The authors sought to compare and contrast the clinical criterion standards currently used in a cohort of emergency department (ED) patients to diagnose acute heart failure syndromes (AHFS). In a prospective observational study of patients with signs and symptoms of AHFS, 3 criterion standards were examined: (1) the treating ED physician's diagnosis; (2) the hospital discharge diagnosis; and (3) a diagnosis based on medical record review by a panel of cardiologists. Using Cohen's kappa (κ) coefficient, the authors assessed agreement and then compared the different standards by repeatedly setting one as the criterion standard and the other two as index tests. A total of 483 patients were enrolled. Across all criterion standards, patients with AHFS were more likely to have a history of AHFS, congestion on physical examination and chest radiography, and elevated natriuretic peptide levels than those without AHFS. The standards agreed well (cardiology review vs hospital discharge diagnosis, κ=0.74; cardiology review vs ED diagnosis, κ=0.66; ED diagnosis vs hospital discharge diagnosis κ=0.59). Each method had similar sensitivity but differing specificities. Different criterion standards identify different patients from among those being evaluated for AHFS. Researchers should consider this when choosing between the various criterion standard approaches when evaluating new index tests.
© 2012 Wiley Periodicals, Inc.