External Validation of the eRADAR Risk Score for Detecting Undiagnosed Dementia in Two Real-World Healthcare Systems

R Yates Coley; Julia J Smith; Leah Karliner; Abisola E Idu; Sei J Lee; Sharon Fuller; Rosemary Lam; Deborah E Barnes; Sascha Dublin

doi:10.1007/s11606-022-07736-6

External Validation of the eRADAR Risk Score for Detecting Undiagnosed Dementia in Two Real-World Healthcare Systems

J Gen Intern Med. 2023 Feb;38(2):351-360. doi: 10.1007/s11606-022-07736-6. Epub 2022 Jul 29.

Authors

R Yates Coley^{1

2}, Julia J Smith³, Leah Karliner^{4

5}, Abisola E Idu³, Sei J Lee^{6

7}, Sharon Fuller³, Rosemary Lam⁵, Deborah E Barnes^{6

8

9

10}, Sascha Dublin^{3

11}

Affiliations

¹ Kaiser Permanente Washington Health Research Institute, Seattle, WA, USA. Rebecca.Y.Coley@kp.org.
² Department of Biostatistics, School of Public Health, University of Washington, Seattle, WA, USA. Rebecca.Y.Coley@kp.org.
³ Kaiser Permanente Washington Health Research Institute, Seattle, WA, USA.
⁴ Multiethnic Health Equity Research Center, University of California San Francisco, San Francisco, USA.
⁵ Division of General Internal Medicine, Department of Medicine, University of California San Francisco, San Francisco, CA, USA.
⁶ San Francisco Veterans Affairs Health Care System, San Francisco, CA, USA.
⁷ Division of Geriatrics, Department of Medicine, University of California San Francisco, San Francisco, CA, USA.
⁸ Weill Institute for Neurosciences, Department of Psychiatry and Behavioral Sciences, University of California, San Francisco, San Francisco, CA, USA.
⁹ Department of Epidemiology & Biostatistics, University of California, San Francisco, San Francisco, CA, USA.
¹⁰ Osher Center for Integrative Health, University of California, San Francisco, San Francisco, CA, USA.
¹¹ Department of Epidemiology, University of Washington, Seattle, WA, USA.

Abstract

Background: Fifty percent of people living with dementia are undiagnosed. The electronic health record (EHR) Risk of Alzheimer's and Dementia Assessment Rule (eRADAR) was developed to identify older adults at risk of having undiagnosed dementia using routinely collected clinical data.

Objective: To externally validate eRADAR in two real-world healthcare systems, including examining performance over time and by race/ethnicity.

Design: Retrospective cohort study PARTICIPANTS: 129,315 members of Kaiser Permanente Washington (KPWA), an integrated health system providing insurance coverage and medical care, and 13,444 primary care patients at University of California San Francisco Health (UCSF), an academic medical system, aged 65 years or older without prior EHR documentation of dementia diagnosis or medication.

Main measures: Performance of eRADAR scores, calculated annually from EHR data (including vital signs, diagnoses, medications, and utilization in the prior 2 years), for predicting EHR documentation of incident dementia diagnosis within 12 months.

Key results: A total of 7631 dementia diagnoses were observed at KPWA (11.1 per 1000 person-years) and 216 at UCSF (4.6 per 1000 person-years). The area under the curve was 0.84 (95% confidence interval: 0.84-0.85) at KPWA and 0.79 (0.76-0.82) at UCSF. Using the 90^th percentile as the cut point for identifying high-risk patients, sensitivity was 54% (53-56%) at KPWA and 44% (38-51%) at UCSF. Performance was similar over time, including across the transition from International Classification of Diseases, version 9 (ICD-9) to ICD-10 codes, and across racial/ethnic groups (though small samples limited precision in some groups).

Conclusions: eRADAR showed strong external validity for detecting undiagnosed dementia in two health systems with different patient populations and differential availability of external healthcare data for risk calculations. In this study, eRADAR demonstrated generalizability from a research sample to real-world clinical populations, transportability across health systems, robustness to temporal changes in healthcare, and similar performance across larger racial/ethnic groups.

Keywords: decision support tools; dementia; early diagnosis; electronic health record data; prediction.

Publication types

Research Support, N.I.H., Extramural
Research Support, U.S. Gov't, P.H.S.

MeSH terms

Aged
Delivery of Health Care*
Dementia* / diagnosis
Humans
Retrospective Studies
Risk Factors
Washington

Abstract

Publication types

MeSH terms

Grants and funding