Pay Rates and Subject Performance in Social Science Experiments Using Crowdsourced Online Samples

Andersen, David J.; Lau, Richard R.

Abstract

Mechanical Turk has become an important source of subjects for social science experiments, providing a low-cost alternative to the convenience of using undergraduates while avoiding the expense of drawing fully representative samples. However, we know little about how the rates we pay to “Turkers” for participating in social science experiments affects their participation. This study examines subject performance using two experiments – a short survey experiment and a longer dynamic process tracing study of political campaigns – that recruited Turkers at different rates of pay. Looking at demographics and using measures of attention, engagement and evaluation of the candidates, we find no effects of pay rates upon subject recruitment or participation. We conclude by discussing implications and ethical standards of pay.

Citation

Andersen, D. J., & Lau, R. R. (2018). Pay Rates and Subject Performance in Social Science Experiments Using Crowdsourced Online Samples. Journal of Experimental Political Science, 5(3), 217-229. https://doi.org/10.1017/xps.2018.7

• Journal Article Type: Article
• Publication Date: 2018
• Deposit Date: Jan 7, 2020
• Publicly Available Date: Jan 12, 2021
• Journal: Journal of Experimental Political Science
• Print ISSN: 2052-2630
• Electronic ISSN: 2052-2649
• Publisher: Cambridge University Press
• Peer Reviewed: Peer Reviewed
• Volume: 5
• Issue: 3
• Pages: 217-229
• DOI: https://doi.org/10.1017/xps.2018.7
• Public URL: https://durham-repository.worktribe.com/output/1274542