Accuracy of the surgical risk preoperative assessment system universal risk calculator in predicting risk for patients undergoing selected operations in 9 specialty areas

Abstract

Background: The universal Surgical Risk Preoperative Assessment System (SURPAS) prediction models for postoperative adverse outcomes have good accuracy for estimating risk in broad surgical populations and for surgical specialties. The accuracy in individual operations has not yet been assessed. The objective of this study was to evaluate the Surgical Risk Preoperative Assessment System in predicting adverse outcomes for selected individual operations.

Methods: The SURPAS models were applied to the top 2 most frequent common procedural terminology codes in 9 surgical specialties and 5 additional common general surgical operations in the 2009 to 2018 database of the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP). Goodness of fit statistics were estimated, including c-indices for discrimination, Hosmer-Lemeshow graphs and P values for calibration, overall observed versus expected event rates, and Brier scores.

Results: The total sample size was 2,020,172, which represented 29% of the 6.9 million operations in the ACS NSQIP database. Average c-indices across 12 outcomes were acceptable (≥0.70) for 13 (56.5%) of the 23 operations. Overall observed-to-expected rates were similar for mortality and overall morbidity across the 23 operations. Hosmer-Lemeshow graphs over quintiles of risk comparing observed-to-expected rates of mortality and overall morbidity were similar for 52% and 70% of operations, respectively. Model performance was better in less complex operations and those done in patients with lower preoperative risk.

Conclusion: SURPAS displayed accuracy in estimating postoperative adverse events for some of the 23 operations studied, but not all. In the procedures where SURPAS was not accurate, developing disease or operation-specific risk models might be appropriate.