Inclusion of nonrandomized studies of interventions in systematic reviews of interventions: updated guidance from the Agency for Health Care Research and Quality Effective Health Care program

Ian J Saldanha; Gaelen P Adam; Lionel L Bañez; Eric B Bass; Elise Berliner; Beth Devine; Noah Hammarlund; Anjali Jain; Susan L Norris; Andrea C Skelly; Kelly Vander Ley; Zhen Wang; Timothy J Wilt; Meera Viswanathan

doi:10.1016/j.jclinepi.2022.08.015

Inclusion of nonrandomized studies of interventions in systematic reviews of interventions: updated guidance from the Agency for Health Care Research and Quality Effective Health Care program

J Clin Epidemiol. 2022 Dec:152:300-306. doi: 10.1016/j.jclinepi.2022.08.015. Epub 2022 Sep 19.

Authors

Affiliations

¹ Center for Evidence Synthesis in Health, Department of Health Services, Policy, and Practice, Department of Epidemiology, Brown University School of Public Health, Providence, RI, USA. Electronic address: ian_saldanha@brown.edu.
² Center for Evidence Synthesis in Health, Department of Health Services, Policy, and Practice, Department of Epidemiology, Brown University School of Public Health, Providence, RI, USA.
³ Evidence-Based Practice Center Program, Center for Evidence and Practice Improvement, Agency for Healthcare Research and Quality, Rockville, MD, USA.
⁴ Department of Medicine, Johns Hopkins School of Medicine, Baltimore, MD, USA.
⁵ Cerner Enviza, Inc., Kansas City, MO, USA.
⁶ Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, University of Washington School of Pharmacy, Seattle, WA, USA.
⁷ Department of Health Services Research, Management, and Policy, University of Florida, Gainesville, FL, USA.
⁸ Oregon Health and Science University, Portland, OR, USA.
⁹ Pacific Northwest Evidence-Based Practice Center, Portland, OR, USA; Aggregate Analytics, Inc., Fircrest, WA, USA.
¹⁰ Department of Medical and Clinical Informatics and Clinical Epidemiology, Oregon Health and Science University, Portland, OR, USA.
¹¹ Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, MN, USA.
¹² Minneapolis VA Center for Care Delivery and Outcomes Research, University of Minnesota Schools of Medicine and Public Health, Minneapolis, MN, USA.
¹³ RTI International, Research Triangle Park, NC, USA.

Abstract

Objectives: We developed guidance to inform decisions regarding the inclusion of nonrandomized studies of interventions (NRSIs) in systematic reviews (SRs) of the effects of interventions.

Study design and setting: The guidance workgroup comprised SR experts and used an informal consensus generation method.

Results: Instead of recommending NRSI inclusion only if randomized controlled trials (RCTs) are insufficient to address the SR key question, different topics may require different decisions regarding NRSI inclusion. We identified important considerations to inform such decisions from topic refinement through protocol development. During topic scoping and refinement, considerations were related to the clinical decisional dilemma, adequacy of RCTs to address the key questions, risk of bias in NRSIs, and the extent to which NRSIs are likely to complement RCTs. When NRSIs are included, during SR team formation, familiarity with topic-specific data sources and advanced analytic methods for NRSIs should be considered. During protocol development, the decision regarding NRSI inclusion or exclusion should be justified, and potential implications explained. When NRSIs are included, the protocol should describe the processes for synthesizing evidence from RCTs and NRSIs and determining the overall strength of evidence.

Conclusion: We identified specific considerations for decisions regarding NRSI inclusion in SRs and highlight the importance of flexibility and transparency.

Keywords: Effectiveness; Guidance; Interventions; Nonrandomized studies; Study eligibility; Systematic reviews.

MeSH terms

Bias
Delivery of Health Care
Health Services Research*
Humans
Research Design*
Systematic Reviews as Topic

Abstract

MeSH terms

Grants and funding