Machine learning-assisted screening for cognitive impairment in the emergency department

Simon R Yadgir; Collin Engstrom; Gwen Costa Jacobsohn; Rebecca K Green; Courtney M C Jones; Jeremy T Cushman; Thomas V Caprio; Amy J H Kind; Michael Lohmeier; Manish N Shah; Brian W Patterson

doi:10.1111/jgs.17491

Machine learning-assisted screening for cognitive impairment in the emergency department

J Am Geriatr Soc. 2022 Mar;70(3):831-837. doi: 10.1111/jgs.17491. Epub 2021 Oct 13.

Authors

Simon R Yadgir¹, Collin Engstrom^{1

2}, Gwen Costa Jacobsohn¹, Rebecca K Green¹, Courtney M C Jones^{3

4}, Jeremy T Cushman^{3

4

5}, Thomas V Caprio⁶, Amy J H Kind^{7

8

9}, Michael Lohmeier¹, Manish N Shah^{1

7

10}, Brian W Patterson^{1

11

12}

Affiliations

¹ BerbeeWalsh Department of Emergency Medicine, University of Wisconsin-Madison School of Medicine and Public Health, Madison, Wisconsin, USA.
² Department of Computer Science, Winona State University, Rochester, MN, USA.
³ Department of Emergency Medicine, University of Rochester Medical Center, Rochester, New York, USA.
⁴ Department of Public Health Sciences, University of Rochester Medical Center, Rochester, New York, USA.
⁵ Department of Health Policy and Management, School of Public Health, University of Maryland, College Park, Maryland, USA.
⁶ Division of Geriatrics, Department of Medicine, University of Rochester Medical Center, Rochester, New York, USA.
⁷ Division of Geriatrics and Gerontology, Department of Medicine, School of Medicine and Public Health, University of Wisconsin, Madison, Wisconsin, USA.
⁸ William S. Middleton Veterans Affairs Geriatrics Research, Education, and Clinical Center, Madison, Wisconsin, USA.
⁹ UW Center for Health Disparities Research, School of Medicine and Public Health, University of Wisconsin-Madison, Madison, Wisconsin, USA.
¹⁰ Department of Population Health Sciences, School of Medicine and Public Health, University of Wisconsin-Madison, Madison, Wisconsin, USA.
¹¹ Health Innovation Program, University of Wisconsin-Madison, Madison, Wisconsin, USA.
¹² Department of Industrial and Systems Engineering, Department of Biostatistics and Medical Informatics, University of Wisconsin-Madison, Madison, Wisconsin, USA.

Abstract

Background/objectives: Despite a high prevalence and association with poor outcomes, screening to identify cognitive impairment (CI) in the emergency department (ED) is uncommon. Identification of high-risk subsets of older adults is a critical challenge to expanding screening programs. We developed and evaluated an automated screening tool to identify a subset of patients at high risk for CI.

Methods: In this secondary analysis of existing data collected for a randomized control trial, we developed machine-learning models to identify patients at higher risk of CI using only variables available in electronic health record (EHR). We used records from 1736 community-dwelling adults age > 59 being discharged from three EDs. Potential CI was determined based on the Blessed Orientation Memory Concentration (BOMC) test, administered in the ED. A nested cross-validation framework was used to evaluate machine-learning algorithms, comparing area under the receiver-operator curve (AUC) as the primary metric of performance.

Results: Based on BOMC scores, 121 of 1736 (7%) participants screened positive for potential CI at the time of their ED visit. The best performing algorithm, an XGBoost model, predicted BOMC positivity with an AUC of 0.72. With a classification threshold of 0.4, this model had a sensitivity of 0.73, a specificity of 0.64, a negative predictive value of 0.97, and a positive predictive value of 0.13. In a hypothetical ED with 200 older adult visits per week, the use of this model would lead to a decrease in the in-person screening burden from 200 to 77 individuals in order to detect 10 of 14 patients who would fail a BOMC.

Conclusion: This study demonstrates that an algorithm based on EHR data can define a subset of patients at higher risk for CI. Incorporating such an algorithm into a screening workflow could allow screening efforts and resources to be focused where they have the most impact.

Keywords: cognitive impairment; delirium; dementia; emergency medicine; machine learning.

Publication types

Randomized Controlled Trial
Research Support, N.I.H., Extramural
Research Support, U.S. Gov't, P.H.S.

MeSH terms

Aged
Cognitive Dysfunction* / diagnosis
Electronic Health Records
Emergency Service, Hospital
Humans
Machine Learning*
Mass Screening

Abstract

Publication types

MeSH terms

Grants and funding