Princesses are bigger than Elephants: effect size as a category error in evidence based education

Simpson, A.

Abstract

Much of the evidential basis for recent policy decisions is grounded in effect size: the standardised mean difference in outcome scores between a study's intervention and comparison groups. This is interpreted as measuring educational influence, importance or effectiveness of the intervention. This article shows this is a category error at two levels. At the individual study level, the intervention plays only a partial role in effect size, so treating effect size as a measure of the intervention is a mistake. At the meta‐analytic level, the assumptions needed for a valid comparison of the relative effectiveness of interventions on the basis of relative effect size are absurd. While effect size continues to have a role in research design, as a measure of the clarity of a study, policy makers should recognise the lack of a valid role for it in practical decision‐making.

Citation

Simpson, A. (2018). Princesses are bigger than Elephants: effect size as a category error in evidence based education. British Educational Research Journal, 44(5), 897-913. https://doi.org/10.1002/berj.3474

• Journal Article Type: Article
• Acceptance Date: Aug 21, 2018
• Online Publication Date: Sep 19, 2018
• Publication Date: Oct 1, 2018
• Deposit Date: Aug 22, 2018
• Publicly Available Date: Mar 19, 2020
• Journal: British Educational Research Journal
• Print ISSN: 0141-1926
• Electronic ISSN: 1469-3518
• Publisher: Wiley
• Peer Reviewed: Peer Reviewed
• Volume: 44
• Issue: 5
• Pages: 897-913
• DOI: https://doi.org/10.1002/berj.3474
• Public URL: https://durham-repository.worktribe.com/output/1351265