Why use the ANOVA over a t-test?
March 25, 2012
The point of conducting an experiment is to find a significant effect between the stimuli being tested. To do this various statistical tests are used, the 2 being discussed in this blog will be the ANOVA and the t-test. In a psychology experiment an independent variable and dependant variable are the stimuli being manipulated and the behaviour being measured. Statistical tests are carried out to confirm if the behaviour occurring is more than chance.
The t-test compares the means between 2 samples and is simple to conduct, but if there is more than 2 conditions in an experiment a ANOVA is required. The fact the ANOVA can test more than one treatment is a major advantage over other statistical analysis such as the t-test, it opens up many testing capabilities but it certainly doesn’t help with mathematical headaches. It is important to know that when looking at the analysis of variance an IV is called a factor, the treatment conditions or groups in an experiment are called the levels of the factor. ANOVA’s use an F-ratio as its significance statistic which is variance because it is impossible to calculate the sample means difference with more than two samples.
T-tests are easier to conduct, so why not conduct a t-test for the possible interactions in the experiment? A Type I error is the answer because the more hypothesis tests you use the more you risk making a type I error and the less power a test has. There is no disputing the t-test changed statistics with its ability to find significance with a small sample, but as previously mentioned the ANOVA allowed for testing more than 2 means. ANOVA’s are used a lot professionally when testing pharmaceuticals and therapies.
The ANOVA is an important test because it enables us to see for example how effective two different types of treatment are and how durable they are. Effectively a ANOVA can tell us how well a treatment work, how long it lasts and how budget friendly it will be an example being intensive early behavioural intervention (EIBI) for autistic children which lasts a long time with a lot hour, has amazing results but costs a lot of money. The ANOVA is able to tell us if another therapy can do the same task in shorter amount of time and therefor costing less and making the treatment more accessible. Conducting this test would also help establish concurrent validity for the therapy against EIBI. The F-ratio tells the researcher how big of a difference there is between the conditions and the effect is more than just chance. ANOVA test assumes three things:
- The population sample must be normal
- The observations must be independent in each sample
- The population the samples are selected from have equal variance a.k.a. homogeneity of variance.
These requirements are the same for a paired and a repeated measures t-test and these measured are solved in the same way for the t-test and the ANOVA. The population sample is assumed to be normal anyway, the independent samples is achieved with the design of the experiment, if the variance is not correct then normally more data (participants) is needed in the experiment.
In conclusion it is necessary to use the ANOVA when the design of a study has more than 2 condition to compare. The t-test is simple and less daunting especially when you see a 2x4x5 factorial ANOVA is needed, but the risk of committing a type I error is not worth it. The time you spent conducting the experiment only to have it declared obsolete because the right statistical test wasn’t conducted would be a waste of time and resources, statistical tests should be used correctly for this reason.