Welch's t-test

Student's t-test assumes that the sample means being compared for two populations are normally distributed, and that the populations have equal variances. Welch's t-test is designed for unequal population variances, but the assumption of normality is maintained.^[1] Welch's t-test is an approximate solution to the Behrens–Fisher problem.

Welch's t-test defines the statistic t by the following formula:

${\displaystyle t={\frac {\Delta {\overline {X}}}{s_{\Delta {\bar {X}}}}}={\frac {{\overline {X}}_{1}-{\overline {X}}_{2}}{\sqrt {{s_{{\bar {X}}_{1}}^{2}}+{s_{{\bar {X}}_{2}}^{2}}}}}\,}$ 

${\displaystyle s_{{\bar {X}}_{i}}={s_{i} \over {\sqrt {N_{i}}}}\,}$ 

where ${\displaystyle {\overline {X}}_{i}}$  and ${\displaystyle s_{{\bar {X}}_{i}}}$  are the ${\displaystyle i^{\text{th}}}$  sample mean and its standard error, with ${\displaystyle s_{i}}$  denoting the corrected sample standard deviation, and sample size ${\displaystyle N_{i}}$ . Unlike in Student's t-test, the denominator is not based on a pooled variance estimate.

The degrees of freedom ${\displaystyle \nu }$   associated with this variance estimate is approximated using the Welch–Satterthwaite equation:^[4]

${\displaystyle \nu \quad \approx \quad {\frac {\left(\;{\frac {s_{1}^{2}}{N_{1}}}\;+\;{\frac {s_{2}^{2}}{N_{2}}}\;\right)^{2}}{\quad {\frac {s_{1}^{4}}{N_{1}^{2}\nu _{1}}}\;+\;{\frac {s_{2}^{4}}{N_{2}^{2}\nu _{2}}}\quad }}.}$ 

This expression can be simplified when ${\displaystyle N_{1}=N_{2}}$ :

${\displaystyle \nu \approx {\frac {s_{\Delta {\bar {X}}}^{4}}{\nu _{1}^{-1}s_{{\bar {X}}_{1}}^{4}+\nu _{2}^{-1}s_{{\bar {X}}_{2}}^{4}}}.}$ 

Here, ${\displaystyle \nu _{i}=N_{i}-1}$  is the degrees of freedom associated with the i-th variance estimate.

The statistic is approximately from the t-distribution since we have an approximation of the chi-square distribution. This approximation is better done when both ${\displaystyle N_{1}}$  and ${\displaystyle N_{2}}$  are larger than 5.^[5][6]

Once t and ${\displaystyle \nu }$  have been computed, these statistics can be used with the t-distribution to test one of two possible null hypotheses:

• A two-tailed test, in which the two population means are equal; or
• A one-tailed test, in which one of the population means is greater than or equal to the other.

The approximate degrees of freedom are real numbers ${\displaystyle \left(\nu \in \mathbb {R} ^{+}\right)}$  and used as such in statistics-oriented software, whereas they are rounded down to the nearest integer in spreadsheets.

Based on Welch's t-test, it's possible to also construct a two sided confidence interval for the difference in means (while not having to assume equal variances). This will be by taking:

${\displaystyle CI(\mu _{1}-\mu _{2}):{\overline {X}}_{1}-{\overline {X}}_{2}\pm {\sqrt {{s_{{\bar {X}}_{1}}^{2}}+{s_{{\bar {X}}_{2}}^{2}}}}\times t_{\nu ,1-\alpha /2}}$ 

Based on the above definitions of ${\displaystyle s_{{\bar {X}}_{i}}}$  and ${\displaystyle \nu }$ .

Welch's t-test is more robust than Student's t-test and maintains type I error rates close to nominal for unequal variances and for unequal sample sizes under normality. Furthermore, the power of Welch's t-test comes close to that of Student's t-test, even when the population variances are equal and sample sizes are balanced.^[2] Welch's t-test can be generalized to more than 2-samples,^[7] which is more robust than one-way analysis of variance (ANOVA).

It is not recommended to pre-test for equal variances and then choose between Student's t-test or Welch's t-test.^[8] Rather, Welch's t-test can be applied directly and without any substantial disadvantages to Student's t-test as noted above. Welch's t-test remains robust for skewed distributions and large sample sizes.^[9] Reliability decreases for skewed distributions and smaller samples, where one could possibly perform Welch's t-test.^[10]

• Student's t-test
• Z-test
• Factorial experiment
• One-way analysis of variance
• Hotelling's two-sample T-squared statistic, a multivariate extension of Welch's t-test