Real-world Antimicrobial Stewardship Experience in a Large Academic Medical Center: Using Statistical and Machine Learning Approaches to Identify Intervention "Hotspots" in an Antibiotic Audit and Feedback Program

Katherine E Goodman; Emily L Heil; Kimberly C Claeys; Mary Banoub; Jacqueline T Bork

doi:10.1093/ofid/ofac289

Real-world Antimicrobial Stewardship Experience in a Large Academic Medical Center: Using Statistical and Machine Learning Approaches to Identify Intervention "Hotspots" in an Antibiotic Audit and Feedback Program

Open Forum Infect Dis. 2022 Jun 10;9(7):ofac289. doi: 10.1093/ofid/ofac289. eCollection 2022 Jul.

Authors

Katherine E Goodman¹, Emily L Heil², Kimberly C Claeys², Mary Banoub³, Jacqueline T Bork⁴

Affiliations

¹ Department of Epidemiology and Public Health, University of Maryland School of Medicine, Baltimore, Maryland, USA.
² Department of Pharmacy Practice and Science, University of Maryland School of Pharmacy, Baltimore, Maryland, USA.
³ Department of Pharmacy, University of Maryland Medical Center, Baltimore, Maryland, USA.
⁴ Department of Medicine, University of Maryland School of Medicine, Baltimore, Maryland, USA.

Abstract

Background: Prospective audit with feedback (PAF) is an impactful strategy for antimicrobial stewardship program (ASP) activities. However, because PAF requires reviewing large numbers of antimicrobial orders on a case-by-case basis, PAF programs are highly resource intensive. The current study aimed to identify predictors of ASP intervention (ie, feedback) and to build models to identify orders that can be safely bypassed from review, to make PAF programs more efficient.

Methods: We performed a retrospective cross-sectional study of inpatient antimicrobial orders reviewed by the University of Maryland Medical Center's PAF program between 2017 and 2019. We evaluated the relationship between antimicrobial and patient characteristics with ASP intervention using multivariable logistic regression models. Separately, we built prediction models for ASP intervention using statistical and machine learning approaches and evaluated performance on held-out data.

Results: Across 17 503 PAF reviews, 4219 (24%) resulted in intervention. In adjusted analyses, a clinical pharmacist on the ordering unit or receipt of an infectious disease consult were associated with 17% and 56% lower intervention odds, respectively (adjusted odds ratios [aORs], 0.83 and 0.44; P ≤ .001 for both). Fluoroquinolones had the highest adjusted intervention odds (aOR, 3.22 [95% confidence interval, 2.63-3.96]). A machine learning classifier (C-statistic 0.76) reduced reviews by 49% while achieving 78% sensitivity. A "workflow simplified" regression model that restricted to antimicrobial class and clinical indication variables, 2 strong machine learning-identified predictors, reduced reviews by one-third while achieving 81% sensitivity.

Conclusions: Prediction models substantially reduced PAF review caseloads while maintaining high sensitivities. Our results and approach may offer a blueprint for other ASPs.

Keywords: ASP; antimicrobial stewardship; machine learning; prospective audit with feedback.

Grants and funding

K01 HS028363/HS/AHRQ HHS/United States