RTI uses cookies to offer you the best experience online. By clicking “accept” on this website, you opt in and you agree to the use of cookies. If you would like to know more about how RTI uses cookies and how to manage them please view our Privacy Policy here. You can “opt out” or change your mind by visiting: http://optout.aboutads.info/. Click “accept” to agree.
Increased risks for random errors are common in outcomes graded as high certainty of evidence
Gartlehner, G., Nussbaumer-Streit, B., Wagner, G., Patel, S., Swinson-Evans, T., Dobrescu, A., & Gluud, C. (2019). Increased risks for random errors are common in outcomes graded as high certainty of evidence. Journal of Clinical Epidemiology, 106, 50-59. https://doi.org/10.1016/j.jclinepi.2018.10.009
OBJECTIVES: The aim of article was to assess the risk for random errors in outcomes graded as high certainty of evidence (CoE).
STUDY DESIGN AND SETTING: We randomly selected 100 Cochrane reviews with dichotomous outcomes rated as high CoE using Grading of Recommendations Assessment, Development, and Evaluation. To detect increased risks for random errors, two investigators independently conducted trial sequential analysis using conventional thresholds for type I (α = 0.05) and type II (β = 0.10) errors. We dually regraded all outcomes with increased risks for random errors and conducted multivariate logistic regression analyses to determine predictors of increased risks for random errors.
RESULTS: Overall, 38% (95% confidence interval: 28-47%) of high CoE outcomes had increased risks for random errors. Outcomes assessing harms were more frequently affected than outcomes assessing benefits (47% vs. 12%). Regrading of outcomes with increased random errors showed that 74% should have been downgraded based on current guidance. Regression analyses rendered small absolute risk differences (P = 0.009) and low number of events (P = 0.001) as significant predictors of increased risks for random errors.
CONCLUSION: Decisionmakers need to be aware that outcomes rated as high CoE often have increased risks for false-positive or false-negative findings.
