Professor Jochen Einbeck jochen.einbeck@durham.ac.uk
Professor
A Comparison of Threshold-Free Measures for Assessing the Effectiveness of Educational Interventions
Einbeck, Jochen; Coolen-Maturi, Tahani; Uwimpuhwe, Germaine; Singh, Akansha
Authors
Dr Tahani Coolen-Maturi tahani.maturi@durham.ac.uk
Professor
Germaine Uwimpuhwe germaine.uwimpuhwe@durham.ac.uk
sTATISTICIAN
Dr Akansha Singh akansha.singh@durham.ac.uk
Assistant Professor
Abstract
The effectiveness of educational interventions has traditionally been evaluated using effect size measures which focus on a single feature of the distribution of the outcomes under intervention and control conditions: a (standardized) mean difference. Recently there has been increased interest in methods which assess the information contained in the full distributions of outcomes under intervention and control, providing measures of separation from these distributions which do not depend on arbitrary cutoffs, hence which are “threshold-free”. We investigate the statistical relationship between several concepts of this type, and discuss how they can be used to estimate alternative effect size metrics as well as their uncertainties in the context of multilevel models as commonly used for the analysis of educational data. A specific aim of this paper is to investigate how the recently proposed “gain index” relates to other measures of separation including the Area under the Curve (AUC) and the overlapping index. A simulation study, using data with an educationally motivated structure, is presented to compare the different methodologies.
Citation
Einbeck, J., Coolen-Maturi, T., Uwimpuhwe, G., & Singh, A. (online). A Comparison of Threshold-Free Measures for Assessing the Effectiveness of Educational Interventions. The Journal of Experimental Education, https://doi.org/10.1080/00220973.2024.2405738
| Journal Article Type | Article |
|---|---|
| Acceptance Date | Sep 13, 2024 |
| Online Publication Date | Sep 24, 2024 |
| Deposit Date | Oct 10, 2024 |
| Publicly Available Date | Oct 10, 2024 |
| Journal | The Journal of Experimental Education |
| Print ISSN | 0022-0973 |
| Electronic ISSN | 1940-0683 |
| Publisher | Taylor and Francis Group |
| Peer Reviewed | Peer Reviewed |
| DOI | https://doi.org/10.1080/00220973.2024.2405738 |
| Public URL | https://durham-repository.worktribe.com/output/2953384 |
Files
Published Journal Article (Advance Online Version)
(1.9 Mb)
PDF
Publisher Licence URL
http://creativecommons.org/licenses/by/4.0/
You might also like
Simultaneous linear dimension reduction and clustering with flexible variance matrices
(2022)
Presentation / Conference Contribution
Bayesian Adaptive Selection Under Prior Ignorance
(2021)
Presentation / Conference Contribution
A sensitivity analysis and error bounds for the adaptive lasso
(2020)
Presentation / Conference Contribution
Downloadable Citations
About Durham Research Online (DRO)
Administrator e-mail: dro.admin@durham.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search