Significance testing is still wrong, and damages real lives: a brief reply to Spreckelsen and van der Horst, and Nicholson and McCusker

Gorard, S.

Abstract

This paper is a brief reply to two responses to a paper I published previously in this journal. In that first paper I presented a summary of part of the long-standing literature critical of the use of significance testing in real-life research, and reported again on how significance testing is abused, leading to invalid and therefore potentially damaging research outcomes. I illustrated and explained the inverse logic error that is routinely used in significance testing, and argued that all of this should now cease. Although clearly disagreeing with me, neither of the responses to my paper addressed these issues head on. One focussed mainly on arguing with things I had not said (such as that there are no other problems in social science). The other tried to argue either that the inverse logic error is not prevalent, or that there is some other unspecified way of presenting the results of significance testing that does not involve this error. This reply paper summarises my original points, deals with each response paper in turn, and then turns to an examination of how the responders use significance testing in practice in their own studies. All of them use significance testing exactly as I described in the original paper – with non-random cases, and using the probability of the observed data erroneously as though it were the probability of the hypothesis assumed in order to calculate the probability of the observed data.

Citation

Gorard, S. (2017). Significance testing is still wrong, and damages real lives: a brief reply to Spreckelsen and van der Horst, and Nicholson and McCusker. Sociological Research Online, 22(2), Article 11. https://doi.org/10.5153/sro.4281

• Journal Article Type: Article
• Acceptance Date: Mar 15, 2017
• Online Publication Date: May 31, 2017
• Publication Date: May 31, 2017
• Deposit Date: Mar 16, 2017
• Publicly Available Date: May 31, 2018
• Journal: Sociological Research Online
• Electronic ISSN: 1360-7804
• Publisher: SAGE Publications
• Peer Reviewed: Peer Reviewed
• Volume: 22
• Issue: 2
• Article Number: 11
• DOI: https://doi.org/10.5153/sro.4281
• Public URL: https://durham-repository.worktribe.com/output/1390852