9 T-tests
This page is a work in progress, but I welcome early feedback.
9.1 Bottom Line Up Front
The t-test is a special tool for testing mean differences in small sample sizes. Most often it is used to compare means between two groups, but can also be used to test a sample mean against some constant, usually a population mean, or to test differences betweent two dependent samples, such as the same people assessed at two different times. While the t-test is valuable to understand the statistical significance of two means, it should be used in conjunction with a measure of effect size, in the case of the t-test, Cohen’s d.
9.2 Key Concepts
The t-test is used to test for significant differences between a group’s mean and some other mean. It can be used to test between a sample mean and a population mean, the differences between two independent group’s means, or the mean differences between two paired samples. Much like chi-square, an obtained t-value is compared to a critical t-value and if the observed value is greater than the critical value the differences in means are determined to be statistically significant; that is, the difference between the means is larger than would be expected by chance based on the t-distribution.
9.2.1 Student’s t-distribution
The Student’s t distribution is named after an Irish statistician and beer brewer William Sealy Gosset. Gosset wrote under the pseudonym “Student” as he worked for the Guinness brewery in Dublin which required him to publish anonymously. He developed the t-distribution as a way to make reliable inferences from small sample sizes, which was crucial in quality control and brewing processes.
The students_t-distribution is the most commonly used distribution in t-tests. The student’s t-test is used when the sample size is small and the population standard deviation is unknown. It is abell-shaped and symmetric, similar to the normal distribution, but has fatter tails than the normal distribution. The shape of the t-distribution depends on two parameters of the t-test, the degrees of freedom and the noncentrality parameter.
9.2.2 Degrees of Freedom
The t-distribution depends on the degrees of freedom (\(\nu\)) for the particular t-test. Here, degrees_of_freedom refers to the number of values that are free to vary in the estimation of some population parameter. For example, in a one sample t-test, which compares the sample mean to some population mean, one parameter, the population mean, is estimated so the degrees of freedom are the number of values minus one. So in a sample of 34 the degrees of freedom would be 33 for this one-sample t-test. In an independent samples t-test, two population means are estimated, so the degrees of freedom is the sum of the two groups minus two. Assuming a sample of 64 per equal groups for this test would mean there are 126 degrees of freedom. The degrees of freedom for any t-test are important as shape of the assumed t-distribution changes with the degrees of freedom. As the degrees of freedom increase, the t-distribution approaches the shape of the standard normal distribution at around a sample size of 30.
9.2.3 Non-centrality Parameter
The other factor impacting the t-distribution is the noncentrality_parameter (ncp; \(\delta\)) which is the location of the middle of the t-distribution. For most uses of the t-test this location is set to zero, referring to the central t-distribution with a mean of zero. Conversely, noncentral means the middle of the t-distribution is somewhere other than zero. This concept is useful for power analysis in particular, but not often a consideration in a typical t-test.
The t-test tends to be robust to violations of these assumptions when samples from each group are equally-sized and large.
9.3 Types of t-tests
9.4 Independent samples t-test
9.5 One-sample t-test
Comparing sample’s mean to population mean E.g., company’s average employee engagement to all companies’ employee engagement1 ## Cohen’s d Standardized measure of effect size in addition to t-test NHST2 Small: .2, Medium: .5, Large: .8