Background: Hospital quality in vascular surgery is often measured using mortality. We sought to determine whether adjusting mortality for statistical reliability changes hospital quality rankings for vascular surgery.
Methods: Patients undergoing five common vascular surgery procedures (open and endovascular abdominal aortic aneurysm repair, carotid endarterectomy, lower extremity bypass, and aorto-femoral bypass) in the National Surgical Quality Improvement Project (NSQIP) in 2007 were identified (n = 14,559). For each hospital, we first calculated a ratio of observed to expected mortality (O-E ratio) using standard NSQIP techniques. We then adjusted these estimates for statistical noise using empirical Bayes methods, a technique known as reliability adjustment. We then compared rankings based on the standard O-E ratio to the rankings after reliability adjustment.
Results: A total of 172 hospitals reported an average adjusted mortality rate of 2.4% for the five procedures, varying from 0% to 17%. After adjusting for statistical noise using reliability adjustment, hospital mortality was greatly diminished, varying only from 1.7% to 4.1%. This adjustment for reliability had a dramatic effect on hospital rankings. Overall, 43% of hospitals were reclassified into either a higher or lower quartile of performance using traditional methods of risk-adjustment. Fifty-one percent all hospitals in the "best" quartile of performance according to traditional O-E ratios are not classified in the "best" quartile after adjusting for statistical noise. Twenty-six percent of hospitals in the "worst" quartile were no longer classified as such after adjusting for noise.
Conclusions: Adjusting mortality for reliability reduces statistical noise and provides more stable estimates of hospital quality. Reliability adjustment should be standard for comparing hospital quality.
Copyright © 2011 Society for Vascular Surgery. All rights reserved.