| Summary: | Introduction: The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial (RCT). Publication bias has been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. Until recently, outcome reporting bias (ORB) has received less attention. ORB occurs when outcomes are selectively reported for publication based on their results. This can impact upon the results of a meta-analysis, biasing the pooled treatment effect. Methods: Empirical evidence from a series of cohort studies that have assessed study publication bias and ORB in RCTs is reviewed and summarised. Methods for identifying ORB in a review and trial reports are discussed using several motivating examples. Guidelines are obtained and assessed from worldwide organisations and UK based charities that fund RCTs; with regards to trial registration, protocol adherence and trial publication. Statistical, educational and policy solutions are reviewed. A novel weighted linear regression is used to predict missing outcome data when bias is suspected in a review containing structurally related outcomes. A maximum bias bound previously proposed is further assessed using a simulation study to test the sensitivity of the method when heterogeneity is present. The outcome reporting bias in trials (ORBln study is introduced and the impact ORB has on this cohort of reviews from the Cochrane Library is assessed using the bound for maximum bias. Results: Four empirical studies that examined the association between outcome reporting bias and statistical significance found that statistically significant outcomes were more likely to be completely reported than non-significant outcomes (range of odds ratios: 2.2 to 4.7). Guidelines for 73 organisations and charities were reviewed; only eleven of these organisations or charities mentioned the publication of negative as well as positive outcomes. The simulation study indicates that the bound for maximum bias is a useful approach for reviewers to apply to assess the robustness of the conclusions of a review to ORB and can be applied in the presence of heterogeneity. Assessing the impact of ORB in the cohort of reviews in the ORBIT study indicates that ORB had an impact in 15% of the reviews considered. Conclusions: This work summarises the direct empirical evidence for the existence of study publication bias and ORB. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials. There is a need to provide more detailed guidance for those conducting and reporting RCTs to help prevent the selective reporting of results. There are several complementary initiatives to either prevent ORB or to reduce the occurrence of ORB. Therefore, with the implementation and development of these solutions it is hoped that in the future ORB will become less of a problem. However, methods that detect ORB and adjust results for ORB can be useful while the initiatives are given time to make an impact.
|
|---|
