Questionable research practices may have little effect on replicability
This article examines why many studies fail to replicate statistically significant published results. We address this issue within a general statistical framework that also allows us to include various questionable research practices (QRPs) that are thought to reduce replicability. The analyses indi...
Main Authors: | , |
---|---|
Format: | Article |
Language: | English |
Published: |
eLife Sciences Publications Ltd
2020-09-01
|
Series: | eLife |
Subjects: | |
Online Access: | https://elifesciences.org/articles/58237 |
id |
doaj-4250a04349624728a66e927ad62a0aad |
---|---|
record_format |
Article |
spelling |
doaj-4250a04349624728a66e927ad62a0aad2021-05-05T21:31:05ZengeLife Sciences Publications LtdeLife2050-084X2020-09-01910.7554/eLife.58237Questionable research practices may have little effect on replicabilityRolf Ulrich0https://orcid.org/0000-0001-8443-2705Jeff Miller1https://orcid.org/0000-0003-2718-3153Department of Psychology, University of Tübingen, Tübingen, GermanyDepartment of Psychology, University of Otago, Dunedin, New ZealandThis article examines why many studies fail to replicate statistically significant published results. We address this issue within a general statistical framework that also allows us to include various questionable research practices (QRPs) that are thought to reduce replicability. The analyses indicate that the base rate of true effects is the major factor that determines the replication rate of scientific results. Specifically, for purely statistical reasons, replicability is low in research domains where true effects are rare (e.g., search for effective drugs in pharmacology). This point is under-appreciated in current scientific and media discussions of replicability, which often attribute poor replicability mainly to QRPs.https://elifesciences.org/articles/58237meta-researchp-hackingbase rate of true effectsreplicabilityfalse positivesmathematical modelling of research process |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Rolf Ulrich Jeff Miller |
spellingShingle |
Rolf Ulrich Jeff Miller Questionable research practices may have little effect on replicability eLife meta-research p-hacking base rate of true effects replicability false positives mathematical modelling of research process |
author_facet |
Rolf Ulrich Jeff Miller |
author_sort |
Rolf Ulrich |
title |
Questionable research practices may have little effect on replicability |
title_short |
Questionable research practices may have little effect on replicability |
title_full |
Questionable research practices may have little effect on replicability |
title_fullStr |
Questionable research practices may have little effect on replicability |
title_full_unstemmed |
Questionable research practices may have little effect on replicability |
title_sort |
questionable research practices may have little effect on replicability |
publisher |
eLife Sciences Publications Ltd |
series |
eLife |
issn |
2050-084X |
publishDate |
2020-09-01 |
description |
This article examines why many studies fail to replicate statistically significant published results. We address this issue within a general statistical framework that also allows us to include various questionable research practices (QRPs) that are thought to reduce replicability. The analyses indicate that the base rate of true effects is the major factor that determines the replication rate of scientific results. Specifically, for purely statistical reasons, replicability is low in research domains where true effects are rare (e.g., search for effective drugs in pharmacology). This point is under-appreciated in current scientific and media discussions of replicability, which often attribute poor replicability mainly to QRPs. |
topic |
meta-research p-hacking base rate of true effects replicability false positives mathematical modelling of research process |
url |
https://elifesciences.org/articles/58237 |
work_keys_str_mv |
AT rolfulrich questionableresearchpracticesmayhavelittleeffectonreplicability AT jeffmiller questionableresearchpracticesmayhavelittleeffectonreplicability |
_version_ |
1721457975429693440 |