Background: Timely referral for specialist evaluation in patients with advanced heart failure (HF) is a Class 1 recommendation. However, the transition from stage C HF to advanced or stage D HF often goes undetected in routine care, resulting in delayed referral and higher mortality rates.
Objectives: The authors sought to develop an augmented intelligence-enabled workflow using machine learning to identify patients with stage D HF and streamline referral.
Methods: We extracted data on HF patients with encounters from January 1, 2007, to November 30, 2020, from a HF registry within a regional, integrated health system. We created an ensemble machine learning model to predict stage C or stage D HF and integrated the results within the electronic health record.
Results: In a retrospective data set of 14,846 patients, the model had a good positive predictive value (60%) and low sensitivity (25%) for identifying stage D HF in a 100-person, physician-reviewed, holdout test set. During prospective implementation of the workflow from April 1, 2021, to February 15, 2022, 416 patients were reviewed by a clinical coordinator, with agreement between the model and the coordinator in 50.3% of stage D predictions. Twenty-four patients have been scheduled for evaluation in a HF clinic, 4 patients started an evaluation for advanced therapies, and 1 patient received a left ventricular assist device.
Conclusions: An augmented intelligence-enabled workflow was integrated into clinical operations to identify patients with advanced HF. Endeavors such as this require a multidisciplinary team with experience in design thinking, informatics, quality improvement, operations, and health information technology, as well as dedicated resources to monitor and improve performance over time.
Keywords: advanced heart failure; artificial intelligence; augmented intelligence; electronic health record; integrated healthcare system; machine learning.