Systematic reviews with language restrictions and no author contact have lower overall credibility: a methodology study

Zhen Wang; Juan P Brito; Apostolos Tsapas; Marcio L Griebeler; Fares Alahdab; Mohammad Hassan Murad

doi:10.2147/CLEP.S78879

Systematic reviews with language restrictions and no author contact have lower overall credibility: a methodology study

Clin Epidemiol. 2015 Mar 31:7:243-7. doi: 10.2147/CLEP.S78879. eCollection 2015.

Authors

Zhen Wang¹, Juan P Brito², Apostolos Tsapas³, Marcio L Griebeler², Fares Alahdab⁴, Mohammad Hassan Murad⁵

Affiliations

¹ Robert D and Patricia E Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, MN, USA ; Division of Health Care Policy and Research, Department of Health Sciences Research, Mayo Clinic, Rochester, MN, USA ; Knowledge and Evaluation Research Unit, Mayo Clinic, Rochester, MN, USA.
² Division of Endocrinology, Diabetes, Metabolism, and Nutrition, Mayo Clinic, Rochester, MN, USA.
³ Aristotle University of Thessaloniki, Thessaloniki, Greece.
⁴ Robert D and Patricia E Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, MN, USA ; Knowledge and Evaluation Research Unit, Mayo Clinic, Rochester, MN, USA.
⁵ Robert D and Patricia E Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, MN, USA ; Knowledge and Evaluation Research Unit, Mayo Clinic, Rochester, MN, USA ; Division of Preventive, Occupational and Aerospace Medicine, Mayo Clinic, Rochester, MN, USA.

Abstract

Background: High-quality systematic reviews (SRs) require rigorous approaches to identify, appraise, select, and synthesize research evidence relevant to a specific question. In this study, we evaluated the association between two steps in the conduct of an SR - restricting the search to English, and author contact for missing data - and the overall credibility of a SR.

Methods: All SRs cited by the Endocrine Society's Clinical Practice Guidelines published from October 2006 through January 2012 were included. The main outcome was the overall A Measurement Tool to Assess Systematic Reviews (AMSTAR) score, as a surrogate of SR credibility. Nonparametric Kruskal-Wallis tests and multivariable linear regression models were used to investigate the association between language restriction, author contact for missing data, and the overall AMSTAR score.

Results: In all, 69 SRs were included in the analysis. Only 31 SRs (45%) reported searching non-English literature, with an average AMSTAR score of 7.90 (standard deviation [SD] =1.64). SRs that reported language restriction received significantly lower AMSTAR scores (mean =5.25, SD =2.32) (P<0.001). Only 30 SRs (43%) reported contacting authors for missing data, and these received, on average, 2.59 more AMSTAR points (SD =1.95) than those who did not (P<0.001). In multivariable analyses, AMSTAR score was significantly correlated with language restriction (beta =-1.31, 95% confidence interval [CI]: -2.62, -0.01, P=0.05) and author contact for missing data (beta =2.16, 95% CI: 0.91, 3.41, P=0.001). However, after adjusting for compliance with reporting guidelines, language restriction was no longer significantly associated with the AMSTAR score.

Conclusion: Fewer than half of the SRs conducted to support the clinical practice guidelines we examined reported contacting study authors or searched non-English literature. SRs that did not conduct these two steps had lower quality scores, suggesting the importance of these two steps for overall SR credibility.

Keywords: evidence-based medicine; quality of evidence; research design; validity.