Student’s T-Test: Don’t Ignore These Secrets
The Students t-test is a statistical analysis that compares the means of two groups to determine their equalities, assuming normality and equal variances.
Did you know that the incorrect use of statistical tests, including the Student’s t-test, is more common than one might think and can lead to drawing wrong conclusions? This is problematic because incorrect conclusions can result in disastrous decisions — to avoid falling into this trap, read this article to the end.
The Problem
The wide availability, simplicity, and ease of understanding of the Students t-test have made it one of the most commonly used statistical tests worldwide.
However, this popularization has brought some issues, such as the incorrect analysis application.
This concern is so significant that several recent systematic review papers have been published — and in particular studies, up to 80% of published articles contain errors in statistical analysis!
Since solutions to humanity’s problems often depend on these studies, it is troubling to think that such a large proportion of them have been analyzed incorrectly and may have led to erroneous conclusions.
The secrets of correctly conducting a Student’s t-test are not complicated — we only need to understand some basic analysis features!
The Solution
The Student’s t-test determines whether the means of two populations are equal. Therefore, carefully following the five steps listed below is essential for a proper and unquestionable application of a t-test.
1. What type of t-test should I use?
Independent samples t-test is the most commonly used type, assessing whether the means of samples collected from two independent populations are equal. For example, we can check if the muscle mass between men and women in a given company is similar.
Paired samples t-test determines whether the mean of a sample collected before an intervention is equal to that of the sample collected — from the same elements — after the intervention. For example, we may want to check if older women’s blood pressure in a city is the same before and after taking a specific medication. Since we will measure each older woman’s blood pressure twice — before and after the intervention — each sample unit is paired.
One sample t-test, the least used type, checks whether the mean of the sample collected from a population is equal to an already known mean. For example, we may wan
2. Student’s T-test sample size calculation
After defining which type of t-test to use, we need to determine how many elements to collect.
Consistent sample sizes often solve many problems in statistics.
The best way to find the appropriate sample size for our test is to use a specific calculation, with a different one for each statistical test.
For the t-test, we need the two studied populations’ approximate or estimated mean and standard deviation.
In the statistical software, we should enter these values in the sample size calculator for the appropriate t-test type. It then returns the correct number of elements to collect.
But where do we obtain the means and standard deviations of the studied populations? Usually from other pilot studies or similar studies!
Check out this article to learn how to calculate sample size!
Golden tip: G*Power is an excellent free statistical package for calculating sample size.
3. The observations must be independent!
The independence of observations assumption is one of the most important, yet often ignored, assumptions in statistics.
The Students t-test and ALL other statistical tests rely on this assumption, which can bias the analysis and generate misleading results when violated.
To avoid this violation, the observations in your dataset must be independent. Therefore, each element of the sample should not show dependence on others.
The concept of randomness is crucial here. A great way to avoid violating the assumption of independence of observations is to randomly collect your data in a universe suitable for your study.
To gain a better grasp of this idea, let’s take a look at the following example:
Suppose we want to identify the genetic profile of inhabitants in a municipality to answer specific questions. After defining the sample size, we must randomly sample individuals in the city to avoid violating the assumption of independence of observations! Using the same example, a sampling method that would violate this assumption would be if we sampled only individuals from the same family to represent the municipality. In this case, the observations would be dependent on each other since the genetic profile is heritable and exhibits a remarkable similarity between families.
4. Normality
The normality of residuals is an assumption applicable to all so-called parametric inferential analyses.
This assumption means that your residuals (errors) must conform to the famous normal distribution — the Gaussian distribution — to use the t-test.
The normal distribution presents a bell-shaped curve, symmetric around its midpoint, unimodal.
But how do we know if our residuals conform to the normal distribution?
The good news is that statistical analyses called normality tests identify this pattern — the main ones are Shapiro-Wilk, Anderson-Darling, and Kolmogorov-Smirnov.
Be cautious when interpreting these tests’ results to avoid misinterpreting their indications.
If the analyses indicate that the assumption of normality has been violated, there are three possibilities:
(a) Use the t-test even with the assumption of normality violated, claiming it is a robust test for these deviations and that we have an adequate sample size.
(b) Try a data transformation and then recheck for normality.
(c) Lastly, use a nonparametric analysis equivalent to the t-test.
We don’t recommend the first possibility. After all, if you plan to use the t-test regardless of the normality of residuals, why perform these analyses?
We usually choose the second option, and if the normality assumption remains violated, we proceed to the third.
Parametric tests and their nonparametric equivalents:
- Independent samples t-test equivalent is the Mann-Whitney U-test.
- Paired samples t-test equivalent is the Wilcoxon signed-rank test.
- One sample t-test equivalent is the one-sample Wilcoxon signed-rank test.
5. Homoscedasticity
We only need to proceed to this final step if the residuals have not violated the normality assumption. Therefore, we will use a Student’s t-test.
Despite its complex name, the meaning of this assumption is simple to understand.
Homoscedasticity means nothing more than similar variances.
To apply the t-test in its standard format, the variances of the two sampled populations’ residuals must be similar.
To check whether this assumption is being violated, inferential tests are widely available in statistical software!
The F-test, Bartlett’s, and Levene’s tests are among the most commonly used tests to verify the equality of variances. However, it only makes sense to apply them if the residuals have not violated the normality assumption.
This assumption usually poses little problem. If it is being violated — the populations present unequal variances or heteroscedasticity — a minor correction in the Students t-test resolves the issue.
Most statistical software will automatically return the t-test value corrected for heteroscedasticity.
Carefully review the results of the Students t-test and homoscedasticity analyses for correct interpretation.
Concluding Remarks
The Student’s t-test is one of the most widely used inferential statistical analyses worldwide.
Many studies have misapplied it, casting doubt on its results and conclusions.
You can perform an unquestionable analysis by following the five straightforward steps.
To recap, we must:
[1] choose the t-test type, [2] calculate the sample size, [3] conduct proper random sampling, [4] check the normality of the residuals, and [5] verify the equality of variances between populations.
After completing these steps, all that remains is to run the analysis!
If you still need to figure out how to interpret the p-value, check out this excellent article!
