Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature

Abstract Background Preprint usage is growing rapidly in the life sciences; however, questions remain on the relative quality of preprints when compared to published articles. An objective dimension of quality that is readily measurable is completeness of reporting, as transparency can improve the r...

Full description

Bibliographic Details
Main Authors: Clarissa F. D. Carneiro, Victor G. S. Queiroz, Thiago C. Moulin, Carlos A. M. Carvalho, Clarissa B. Haas, Danielle Rayêe, David E. Henshall, Evandro A. De-Souza, Felippe E. Amorim, Flávia Z. Boos, Gerson D. Guercio, Igor R. Costa, Karina L. Hajdu, Lieve van Egmond, Martin Modrák, Pedro B. Tan, Richard J. Abdill, Steven J. Burgess, Sylvia F. S. Guerra, Vanessa T. Bortoluzzi, Olavo B. Amaral
Format: Article
Language:English
Published: BMC 2020-12-01
Series:Research Integrity and Peer Review
Subjects:
Online Access:https://doi.org/10.1186/s41073-020-00101-3
id doaj-3f111683ecee44ac9e14acf2cc92366c
record_format Article
collection DOAJ
language English
format Article
sources DOAJ
author Clarissa F. D. Carneiro
Victor G. S. Queiroz
Thiago C. Moulin
Carlos A. M. Carvalho
Clarissa B. Haas
Danielle Rayêe
David E. Henshall
Evandro A. De-Souza
Felippe E. Amorim
Flávia Z. Boos
Gerson D. Guercio
Igor R. Costa
Karina L. Hajdu
Lieve van Egmond
Martin Modrák
Pedro B. Tan
Richard J. Abdill
Steven J. Burgess
Sylvia F. S. Guerra
Vanessa T. Bortoluzzi
Olavo B. Amaral
spellingShingle Clarissa F. D. Carneiro
Victor G. S. Queiroz
Thiago C. Moulin
Carlos A. M. Carvalho
Clarissa B. Haas
Danielle Rayêe
David E. Henshall
Evandro A. De-Souza
Felippe E. Amorim
Flávia Z. Boos
Gerson D. Guercio
Igor R. Costa
Karina L. Hajdu
Lieve van Egmond
Martin Modrák
Pedro B. Tan
Richard J. Abdill
Steven J. Burgess
Sylvia F. S. Guerra
Vanessa T. Bortoluzzi
Olavo B. Amaral
Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature
Research Integrity and Peer Review
Quality of reporting
Preprint
Peer review
Publication
bioRxiv
Scientific journal
author_facet Clarissa F. D. Carneiro
Victor G. S. Queiroz
Thiago C. Moulin
Carlos A. M. Carvalho
Clarissa B. Haas
Danielle Rayêe
David E. Henshall
Evandro A. De-Souza
Felippe E. Amorim
Flávia Z. Boos
Gerson D. Guercio
Igor R. Costa
Karina L. Hajdu
Lieve van Egmond
Martin Modrák
Pedro B. Tan
Richard J. Abdill
Steven J. Burgess
Sylvia F. S. Guerra
Vanessa T. Bortoluzzi
Olavo B. Amaral
author_sort Clarissa F. D. Carneiro
title Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature
title_short Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature
title_full Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature
title_fullStr Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature
title_full_unstemmed Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature
title_sort comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature
publisher BMC
series Research Integrity and Peer Review
issn 2058-8615
publishDate 2020-12-01
description Abstract Background Preprint usage is growing rapidly in the life sciences; however, questions remain on the relative quality of preprints when compared to published articles. An objective dimension of quality that is readily measurable is completeness of reporting, as transparency can improve the reader’s ability to independently interpret data and reproduce findings. Methods In this observational study, we initially compared independent samples of articles published in bioRxiv and in PubMed-indexed journals in 2016 using a quality of reporting questionnaire. After that, we performed paired comparisons between preprints from bioRxiv to their own peer-reviewed versions in journals. Results Peer-reviewed articles had, on average, higher quality of reporting than preprints, although the difference was small, with absolute differences of 5.0% [95% CI 1.4, 8.6] and 4.7% [95% CI 2.4, 7.0] of reported items in the independent samples and paired sample comparison, respectively. There were larger differences favoring peer-reviewed articles in subjective ratings of how clearly titles and abstracts presented the main findings and how easy it was to locate relevant reporting information. Changes in reporting from preprints to peer-reviewed versions did not correlate with the impact factor of the publication venue or with the time lag from bioRxiv to journal publication. Conclusions Our results suggest that, on average, publication in a peer-reviewed journal is associated with improvement in quality of reporting. They also show that quality of reporting in preprints in the life sciences is within a similar range as that of peer-reviewed articles, albeit slightly lower on average, supporting the idea that preprints should be considered valid scientific contributions.
topic Quality of reporting
Preprint
Peer review
Publication
bioRxiv
Scientific journal
url https://doi.org/10.1186/s41073-020-00101-3
work_keys_str_mv AT clarissafdcarneiro comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature
AT victorgsqueiroz comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature
AT thiagocmoulin comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature
AT carlosamcarvalho comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature
AT clarissabhaas comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature
AT daniellerayee comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature
AT davidehenshall comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature
AT evandroadesouza comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature
AT felippeeamorim comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature
AT flaviazboos comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature
AT gersondguercio comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature
AT igorrcosta comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature
AT karinalhajdu comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature
AT lievevanegmond comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature
AT martinmodrak comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature
AT pedrobtan comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature
AT richardjabdill comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature
AT stevenjburgess comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature
AT sylviafsguerra comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature
AT vanessatbortoluzzi comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature
AT olavobamaral comparingqualityofreportingbetweenpreprintsandpeerreviewedarticlesinthebiomedicalliterature
_version_ 1724399177421553664
spelling doaj-3f111683ecee44ac9e14acf2cc92366c2020-12-06T12:08:00ZengBMCResearch Integrity and Peer Review2058-86152020-12-015111910.1186/s41073-020-00101-3Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literatureClarissa F. D. Carneiro0Victor G. S. Queiroz1Thiago C. Moulin2Carlos A. M. Carvalho3Clarissa B. Haas4Danielle Rayêe5David E. Henshall6Evandro A. De-Souza7Felippe E. Amorim8Flávia Z. Boos9Gerson D. Guercio10Igor R. Costa11Karina L. Hajdu12Lieve van Egmond13Martin Modrák14Pedro B. Tan15Richard J. Abdill16Steven J. Burgess17Sylvia F. S. Guerra18Vanessa T. Bortoluzzi19Olavo B. Amaral20Institute of Medical Biochemistry Leopoldo de Meis, Federal University of Rio de JaneiroInstitute of Medical Biochemistry Leopoldo de Meis, Federal University of Rio de JaneiroInstitute of Medical Biochemistry Leopoldo de Meis, Federal University of Rio de JaneiroSeção de Arbovirologia e Febres Hemorrágicas, Instituto Evandro ChagasDepartamento de Bioquímica, Instituto de Ciências Básicas da Saúde, Universidade Federal do Rio Grande do SulBiomedical Sciences Institute, Federal University of Rio de JaneiroUniversity of Edinburgh Medical SchoolInstitute of Medical Biochemistry Leopoldo de Meis, Federal University of Rio de JaneiroInstitute of Medical Biochemistry Leopoldo de Meis, Federal University of Rio de JaneiroPrograma de Pós-Graduação em Psicobiologia, Universidade Federal de São PauloDepartment of Psychiatry, University of MinnesotaInstitute of Medical Biochemistry Leopoldo de Meis, Federal University of Rio de JaneiroBiomedical Sciences Institute, Federal University of Rio de JaneiroDepartment of Neuroscience, Uppsala UniversityInstitute of Microbiology of the Czech Academy of SciencesBiomedical Sciences Institute, Federal University of Rio de JaneiroDepartment of Genetics, Cell Biology, and Development, University of MinnesotaCarl R Woese Institute for Genomic Biology, University of Illinois at Urbana-ChampaignCentro Universitário Metropolitano da Amazônia, Instituto Euro-Americano de Educação, Ciência e TecnologiaDepartamento de Bioquímica, Instituto de Ciências Básicas da Saúde, Universidade Federal do Rio Grande do SulInstitute of Medical Biochemistry Leopoldo de Meis, Federal University of Rio de JaneiroAbstract Background Preprint usage is growing rapidly in the life sciences; however, questions remain on the relative quality of preprints when compared to published articles. An objective dimension of quality that is readily measurable is completeness of reporting, as transparency can improve the reader’s ability to independently interpret data and reproduce findings. Methods In this observational study, we initially compared independent samples of articles published in bioRxiv and in PubMed-indexed journals in 2016 using a quality of reporting questionnaire. After that, we performed paired comparisons between preprints from bioRxiv to their own peer-reviewed versions in journals. Results Peer-reviewed articles had, on average, higher quality of reporting than preprints, although the difference was small, with absolute differences of 5.0% [95% CI 1.4, 8.6] and 4.7% [95% CI 2.4, 7.0] of reported items in the independent samples and paired sample comparison, respectively. There were larger differences favoring peer-reviewed articles in subjective ratings of how clearly titles and abstracts presented the main findings and how easy it was to locate relevant reporting information. Changes in reporting from preprints to peer-reviewed versions did not correlate with the impact factor of the publication venue or with the time lag from bioRxiv to journal publication. Conclusions Our results suggest that, on average, publication in a peer-reviewed journal is associated with improvement in quality of reporting. They also show that quality of reporting in preprints in the life sciences is within a similar range as that of peer-reviewed articles, albeit slightly lower on average, supporting the idea that preprints should be considered valid scientific contributions.https://doi.org/10.1186/s41073-020-00101-3Quality of reportingPreprintPeer reviewPublicationbioRxivScientific journal