Self-assessment of team performance using T-NOTECHS in simulated pediatric trauma resuscitation is not consistent with expert assessment

Minna M Wieck; Cory McLaughlin; Todd P Chang; Alyssa Rake; Caron Park; Christianne Lane; Rita V Burke; L Caulette Young; Elizabeth A Cleek; Inge Morton; Catherine J Goodhue; Randall S Burd; Henri R Ford; Jeffrey S Upperman; Aaron R Jensen

doi:10.1016/j.amjsurg.2018.01.010

Self-assessment of team performance using T-NOTECHS in simulated pediatric trauma resuscitation is not consistent with expert assessment

Am J Surg. 2018 Sep;216(3):630-635. doi: 10.1016/j.amjsurg.2018.01.010. Epub 2018 Feb 1.

Authors

Minna M Wieck¹, Cory McLaughlin², Todd P Chang³, Alyssa Rake⁴, Caron Park⁵, Christianne Lane⁶, Rita V Burke⁷, L Caulette Young⁸, Elizabeth A Cleek⁹, Inge Morton¹⁰, Catherine J Goodhue¹¹, Randall S Burd¹², Henri R Ford¹³, Jeffrey S Upperman¹⁴, Aaron R Jensen¹⁵

Affiliations

¹ Division of Pediatric Surgery, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: wieck@ohsu.edu.
² Division of Pediatric Surgery, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: cmclaughlin@chla.usc.edu.
³ Division of Emergency and Transport Medicine, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: tochang@chla.usc.edu.
⁴ Division of Anesthesiology and Critical Care Medicine, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA; Las Madrinas Simulation Research Laboratory, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: arake@chla.usc.edu.
⁵ Southern California Clinical Translational Science Institute (SC-CTSI), Department of Preventative Medicine, Keck School of Medicine, University of Southern California, Los Angeles, CA 90089, USA. Electronic address: caronpar@usc.edu.
⁶ Southern California Clinical Translational Science Institute (SC-CTSI), Department of Preventative Medicine, Keck School of Medicine, University of Southern California, Los Angeles, CA 90089, USA. Electronic address: christianne.lane@med.usc.edu.
⁷ Trauma Program, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA; Division of Pediatric Surgery, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: riburke@chla.usc.edu.
⁸ Division of Anesthesiology and Critical Care Medicine, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA; Las Madrinas Simulation Research Laboratory, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: lyoung@chla.usc.edu.
⁹ Trauma Program, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA; Division of Pediatric Surgery, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: elizabeth.cleek@marquette.edu.
¹⁰ Division of Emergency and Transport Medicine, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: imorton@chla.usc.edu.
¹¹ Trauma Program, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA; Division of Pediatric Surgery, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: cgoodhue@chla.usc.edu.
¹² Division of Trauma and Burn Surgery, Children's National Medical Center, Washington, DC 20010, USA. Electronic address: rburd@childrensnational.org.
¹³ Trauma Program, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA; Division of Pediatric Surgery, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: hford@chla.usc.edu.
¹⁴ Trauma Program, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA; Division of Pediatric Surgery, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: jupperman@chla.usc.edu.
¹⁵ Trauma Program, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA; Division of Pediatric Surgery, Children's Hospital Los Angeles, Keck School of Medicine, University of Southern California, Los Angeles, CA 90027, USA. Electronic address: ajensen@chla.usc.edu.

Abstract

Background: The Trauma NOn-TECHnical Skills (T-NOTECHS) tool has been used to assess teamwork in trauma resuscitation, but its reliability and validity for self-assessment is unknown. Our purpose was to determine the reliability and validity of self-administered T-NOTECHS in pediatric trauma resuscitation.

Methods: Simulated in situ resuscitations were evaluated using T-NOTECHS in real time by experts and immediately afterwards by team members. Reliability was analyzed with linear-weighted kappa and intra-class correlation. T-NOTECHS scores were compared between expert (gold-standard) and self-assessment.

Results: Fifteen simulations were examined. T-NOTECHS scores were similar between self- and expert assessment for leadership. Self-assessment scores were higher than expert for the other domains and total composite score. Inter-rater reliability for total score was similar between the two groups, but differences were observed in the domains.

Conclusions: Self-assessment is not interchangeable with expert rating when using T-NOTECHS. Future studies need to determine how self-assessment can be best utilized.

Level of evidence: Studies of diagnostic accuracy - Level 2.

Publication types

Research Support, N.I.H., Extramural
Research Support, Non-U.S. Gov't

MeSH terms

Child
Clinical Competence*
Humans
Leadership*
Patient Care Team / standards*
Patient Simulation
Reproducibility of Results
Resuscitation / education*
Self-Assessment*
Trauma Centers*
Traumatology / education*

Abstract

Publication types

MeSH terms

Grants and funding