Validation study of the ACS NSQIP surgical risk calculator for two procedures in Japan

Meixi Ma; Yaoming Liu; Mitsukazu Gotoh; Arata Takahashi; Shigeru Marubashi; Yasuyuki Seto; Itaru Endo; Clifford Y Ko; Mark E Cohen

doi:10.1016/j.amjsurg.2021.06.008

Validation study of the ACS NSQIP surgical risk calculator for two procedures in Japan

Am J Surg. 2021 Nov;222(5):877-881. doi: 10.1016/j.amjsurg.2021.06.008. Epub 2021 Jun 23.

Authors

Meixi Ma¹, Yaoming Liu², Mitsukazu Gotoh³, Arata Takahashi⁴, Shigeru Marubashi³, Yasuyuki Seto³, Itaru Endo⁵, Clifford Y Ko⁶, Mark E Cohen²

Affiliations

¹ American College of Surgeons, Division of Research and Optimal Patient Care, 633 N. Saint Clair St., Chicago, IL, 60611-3211, USA; University of Alabama at Birmingham, Department of Surgery, 1720 2nd Ave S, BDB 202, Birmingham, AL, 35294-0012, USA. Electronic address: mma@facs.org.
² American College of Surgeons, Division of Research and Optimal Patient Care, 633 N. Saint Clair St., Chicago, IL, 60611-3211, USA.
³ The Japanese Society of Gastroenterological Surgery, 1-17, Mita, Minato-ku, Tokyo, 108-0073 Japan.
⁴ Department of Healthcare Quality Assessment, The University of Tokyo, 7-3-1 Hongo Bunkyo-ku, Tokyo, 113-0033, Japan.
⁵ Japanese Society of Hepato-Biliary-Pancreatic Surgery, 1-15 Sumiyoshi-cho, Shinjyuku-ku, Tokyo, 162-0065, Japan.
⁶ American College of Surgeons, Division of Research and Optimal Patient Care, 633 N. Saint Clair St., Chicago, IL, 60611-3211, USA; Department of Surgery, University of California Los Angeles David Geffen School of Medicine, 10833 LeConte Avenue., 72-215 CHS, Los Angeles, CA, 90095, USA; VA Greater Los Angeles Healthcare System, 11301 Wiltshire Blvd., Los Angeles, CA, 90073, USA.

PMID: 34175114
DOI: 10.1016/j.amjsurg.2021.06.008

Abstract

Introduction: The ACS NSQIP Surgical Risk Calculator (SRC) assesses risk to support goal-concordant care. While it accurately predicts US outcomes, its performance internationally is unknown. This study evaluates SRC accuracy in predicting mortality following low anterior resection (LAR) and pancreaticoduodenectomy (PD) in NSQIP patients and accuracy retention when applied to native Japanese patients (National Clinical Database, NCD).

Methods: NSQIP (41,260 LAR; 15,114 PD) and NCD cases (61,220 LAR; 27,901 PD) from 2015 to 2017 were processed through the SRC mortality model. Country-specific calibration and discrimination were assessed with and without an intercept correction applied to the Japanese data.

Results: The SRC exhibited acceptable calibration for LAR and PD when applied to NSQIP data but miscalibration for NCD data. A simple correction to the model intercept, motivated by lower mortality rates in the Japanese data, successfully remediated the miscalibration.

Conclusions: The SRC may inaccurately predict surgical risk when applied to the native Japanese population. An intercept correction method is suggested when miscalibration is encountered; it is simple to implement and may permit effective international use of the SRC.

Keywords: ACS NSQIP; National cross-validation; Surgical risk calculator.

MeSH terms

Aged
Aged, 80 and over
Female
Humans
Japan
Male
Pancreaticoduodenectomy / adverse effects*
Pancreaticoduodenectomy / standards
Pancreaticoduodenectomy / statistics & numerical data
Proctectomy / adverse effects*
Proctectomy / standards
Proctectomy / statistics & numerical data
Quality Assurance, Health Care
Reproducibility of Results
Risk Assessment / methods*
Risk Assessment / standards