Psychomotor skill and decision-making efficiency in surgical wire navigation can be objectively evaluated by analysis of intraoperative fluoroscopic image sequences. Prior work suggests that such image-based behavior analysis of operating room (OR) performance can predict performer experience level (R2 = 0.62) and agree with expert opinion (the current standard) on the quality of a final implant construct (R2 = 0.59). However, it is unclear how objective image-based evaluation compares with expert assessments for entire technical OR performances. This study examines the relationships between three key variables: (1) objective image-based criteria, (2) expert opinions, and (3) performing surgeon experience level. A paired-comparison survey of seven experts, based upon eight OR fluoroscopic wire navigation image sequences, shows that the experts' preferences are best explained by objective metrics that reflect psychomotor and decision-making behaviors which are counter-productive to successful implant placement, like image count (R2 = 0.83) and behavior tally (R2 = 0.74). One such behavior, adjustments away from goal, uniquely correlated well with all three key variables: a fluoroscopic image-based analysis composite score (R2 = 0.40), expert consensus (R2 = 0.76), and performer experience (R2 = 0.41). These results confirm that experts view less efficient technical behavior as indicative of lesser technical proficiency. While expert assessments of technical skill were reliable and consistent, neither individual nor consensus expert opinion appears to correlate with performer experience (R2 = 0.11).
Keywords: expert opinion; fluoroscopic image analysis; paired comparison; surgical data science; technical skill assessment.
© 2023 The Authors. Journal of Orthopaedic Research® published by Wiley Periodicals LLC on behalf of Orthopaedic Research Society.