Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature
Abstract Background Preprint usage is growing rapidly in the life sciences; however, questions remain on the relative quality of preprints when compared to published articles. An objective dimension of quality that is readily measurable is completeness of reporting, as transparency can improve the r...
Main Authors: | , , , , , , , , , , , , , , , , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
BMC
2020-12-01
|
Series: | Research Integrity and Peer Review |
Subjects: | |
Online Access: | https://doi.org/10.1186/s41073-020-00101-3 |
id |
doaj-3f111683ecee44ac9e14acf2cc92366c |
---|---|
record_format |
Article |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Clarissa F. D. Carneiro Victor G. S. Queiroz Thiago C. Moulin Carlos A. M. Carvalho Clarissa B. Haas Danielle Rayêe David E. Henshall Evandro A. De-Souza Felippe E. Amorim Flávia Z. Boos Gerson D. Guercio Igor R. Costa Karina L. Hajdu Lieve van Egmond Martin Modrák Pedro B. Tan Richard J. Abdill Steven J. Burgess Sylvia F. S. Guerra Vanessa T. Bortoluzzi Olavo B. Amaral |
spellingShingle |
Clarissa F. D. Carneiro Victor G. S. Queiroz Thiago C. Moulin Carlos A. M. Carvalho Clarissa B. Haas Danielle Rayêe David E. Henshall Evandro A. De-Souza Felippe E. Amorim Flávia Z. Boos Gerson D. Guercio Igor R. Costa Karina L. Hajdu Lieve van Egmond Martin Modrák Pedro B. Tan Richard J. Abdill Steven J. Burgess Sylvia F. S. Guerra Vanessa T. Bortoluzzi Olavo B. Amaral Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature Research Integrity and Peer Review Quality of reporting Preprint Peer review Publication bioRxiv Scientific journal |
author_facet |
Clarissa F. D. Carneiro Victor G. S. Queiroz Thiago C. Moulin Carlos A. M. Carvalho Clarissa B. Haas Danielle Rayêe David E. Henshall Evandro A. De-Souza Felippe E. Amorim Flávia Z. Boos Gerson D. Guercio Igor R. Costa Karina L. Hajdu Lieve van Egmond Martin Modrák Pedro B. Tan Richard J. Abdill Steven J. Burgess Sylvia F. S. Guerra Vanessa T. Bortoluzzi Olavo B. Amaral |
author_sort |
Clarissa F. D. Carneiro |
title |
Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature |
title_short |
Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature |
title_full |
Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature |
title_fullStr |
Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature |
title_full_unstemmed |
Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature |
title_sort |
comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature |
publisher |
BMC |
series |
Research Integrity and Peer Review |
issn |
2058-8615 |
publishDate |
2020-12-01 |
description |
Abstract Background Preprint usage is growing rapidly in the life sciences; however, questions remain on the relative quality of preprints when compared to published articles. An objective dimension of quality that is readily measurable is completeness of reporting, as transparency can improve the reader’s ability to independently interpret data and reproduce findings. Methods In this observational study, we initially compared independent samples of articles published in bioRxiv and in PubMed-indexed journals in 2016 using a quality of reporting questionnaire. After that, we performed paired comparisons between preprints from bioRxiv to their own peer-reviewed versions in journals. Results Peer-reviewed articles had, on average, higher quality of reporting than preprints, although the difference was small, with absolute differences of 5.0% [95% CI 1.4, 8.6] and 4.7% [95% CI 2.4, 7.0] of reported items in the independent samples and paired sample comparison, respectively. There were larger differences favoring peer-reviewed articles in subjective ratings of how clearly titles and abstracts presented the main findings and how easy it was to locate relevant reporting information. Changes in reporting from preprints to peer-reviewed versions did not correlate with the impact factor of the publication venue or with the time lag from bioRxiv to journal publication. Conclusions Our results suggest that, on average, publication in a peer-reviewed journal is associated with improvement in quality of reporting. They also show that quality of reporting in preprints in the life sciences is within a similar range as that of peer-reviewed articles, albeit slightly lower on average, supporting the idea that preprints should be considered valid scientific contributions. |
topic |
Quality of reporting Preprint Peer review Publication bioRxiv Scientific journal |
url |
https://doi.org/10.1186/s41073-020-00101-3 |
work_keys_str_mv |
AT clarissafdcarneiro comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature AT victorgsqueiroz comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature AT thiagocmoulin comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature AT carlosamcarvalho comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature AT clarissabhaas comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature AT daniellerayee comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature AT davidehenshall comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature AT evandroadesouza comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature AT felippeeamorim comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature AT flaviazboos comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature AT gersondguercio comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature AT igorrcosta comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature AT karinalhajdu comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature AT lievevanegmond comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature AT martinmodrak comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature AT pedrobtan comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature AT richardjabdill comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature AT stevenjburgess comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature AT sylviafsguerra comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature AT vanessatbortoluzzi comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature AT olavobamaral comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature |
_version_ |
1724399177421553664 |
spelling |
doaj-3f111683ecee44ac9e14acf2cc92366c2020-12-06T12:08:00ZengBMCResearch Integrity and Peer Review2058-86152020-12-015111910.1186/s41073-020-00101-3Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literatureClarissa F. D. Carneiro0Victor G. S. Queiroz1Thiago C. Moulin2Carlos A. M. Carvalho3Clarissa B. Haas4Danielle Rayêe5David E. Henshall6Evandro A. De-Souza7Felippe E. Amorim8Flávia Z. Boos9Gerson D. Guercio10Igor R. Costa11Karina L. Hajdu12Lieve van Egmond13Martin Modrák14Pedro B. Tan15Richard J. Abdill16Steven J. Burgess17Sylvia F. S. Guerra18Vanessa T. Bortoluzzi19Olavo B. Amaral20Institute of Medical Biochemistry Leopoldo de Meis, Federal University of Rio de JaneiroInstitute of Medical Biochemistry Leopoldo de Meis, Federal University of Rio de JaneiroInstitute of Medical Biochemistry Leopoldo de Meis, Federal University of Rio de JaneiroSeção de Arbovirologia e Febres Hemorrágicas, Instituto Evandro ChagasDepartamento de Bioquímica, Instituto de Ciências Básicas da Saúde, Universidade Federal do Rio Grande do SulBiomedical Sciences Institute, Federal University of Rio de JaneiroUniversity of Edinburgh Medical SchoolInstitute of Medical Biochemistry Leopoldo de Meis, Federal University of Rio de JaneiroInstitute of Medical Biochemistry Leopoldo de Meis, Federal University of Rio de JaneiroPrograma de Pós-Graduação em Psicobiologia, Universidade Federal de São PauloDepartment of Psychiatry, University of MinnesotaInstitute of Medical Biochemistry Leopoldo de Meis, Federal University of Rio de JaneiroBiomedical Sciences Institute, Federal University of Rio de JaneiroDepartment of Neuroscience, Uppsala UniversityInstitute of Microbiology of the Czech Academy of SciencesBiomedical Sciences Institute, Federal University of Rio de JaneiroDepartment of Genetics, Cell Biology, and Development, University of MinnesotaCarl R Woese Institute for Genomic Biology, University of Illinois at Urbana-ChampaignCentro Universitário Metropolitano da Amazônia, Instituto Euro-Americano de Educação, Ciência e TecnologiaDepartamento de Bioquímica, Instituto de Ciências Básicas da Saúde, Universidade Federal do Rio Grande do SulInstitute of Medical Biochemistry Leopoldo de Meis, Federal University of Rio de JaneiroAbstract Background Preprint usage is growing rapidly in the life sciences; however, questions remain on the relative quality of preprints when compared to published articles. An objective dimension of quality that is readily measurable is completeness of reporting, as transparency can improve the reader’s ability to independently interpret data and reproduce findings. Methods In this observational study, we initially compared independent samples of articles published in bioRxiv and in PubMed-indexed journals in 2016 using a quality of reporting questionnaire. After that, we performed paired comparisons between preprints from bioRxiv to their own peer-reviewed versions in journals. Results Peer-reviewed articles had, on average, higher quality of reporting than preprints, although the difference was small, with absolute differences of 5.0% [95% CI 1.4, 8.6] and 4.7% [95% CI 2.4, 7.0] of reported items in the independent samples and paired sample comparison, respectively. There were larger differences favoring peer-reviewed articles in subjective ratings of how clearly titles and abstracts presented the main findings and how easy it was to locate relevant reporting information. Changes in reporting from preprints to peer-reviewed versions did not correlate with the impact factor of the publication venue or with the time lag from bioRxiv to journal publication. Conclusions Our results suggest that, on average, publication in a peer-reviewed journal is associated with improvement in quality of reporting. They also show that quality of reporting in preprints in the life sciences is within a similar range as that of peer-reviewed articles, albeit slightly lower on average, supporting the idea that preprints should be considered valid scientific contributions.https://doi.org/10.1186/s41073-020-00101-3Quality of reportingPreprintPeer reviewPublicationbioRxivScientific journal |