Questionable Metascience Practices

Rubin, Mark

Abstract

Metascientists have studied questionable research practices in science. The present article considers the parallel concept of questionable metascience practices (QMPs). A QMP is a research practice, assumption, or perspective that has been questioned by several commentators as being potentially problematic for metascience and/or the science reform movement. The present article reviews ten QMPs that relate to criticism, replication, bias, generalization, and the characterization of science. Specifically, the following QMPs are considered: (1) rejecting or ignoring self-criticism; (2) a fast ‘n’ bropen scientific criticism style; (3) overplaying the role of replication in science; (4) assuming a replication rate is “too low” without specifying an “acceptable” rate; (5) an unacknowledged metabias towards explaining the replication crisis in terms of researcher bias; (6) assuming that researcher bias can be reduced; (7) devaluing exploratory results as being more “tentative” than confirmatory results; (8) presuming that questionable research practices are problematic research practices; (9) focusing on knowledge accumulation; and (10) focusing on specific scientific methods. It is stressed that only some metascientists engage in some QMPs some of the time, and that these QMPs may not always be problematic. Research is required to estimate the prevalence and impact of QMPs. In the meantime, QMPs should be viewed as invitations to ask questions about how we go about doing better metascience.

Citation

Rubin, M. (2024). Questionable Metascience Practices. Journal of trial and error, 4(1), https://doi.org/10.36850/mr4

• Journal Article Type: Article
• Acceptance Date: Dec 13, 2022
• Online Publication Date: Apr 24, 2023
• Publication Date: May 24, 2024
• Deposit Date: Apr 26, 2023
• Publicly Available Date: Apr 26, 2023
• Journal: Journal of Trial and Error
• Electronic ISSN: 2667-1204
• Peer Reviewed: Peer Reviewed
• Volume: 4
• Issue: 1
• DOI: https://doi.org/10.36850/mr4
• Public URL: https://durham-repository.worktribe.com/output/1175195