Objective: To develop a scoring system that can assess the multidisciplinary management of respiratory failure in a pediatric ICU.
Methods: In a single tertiary pediatric ICU we conducted a simulation-based evaluation in a patient care area auxiliary to the ICU. The subjects were pediatric and emergency medicine residents, nurses, and respiratory therapists who work in the pediatric ICU. A multidisciplinary focus group with experienced providers in pediatric ICU airway management and patient safety specialists was formed. A task-based scoring instrument was developed to evaluate a primary airway provider's performance through Healthcare Failure Mode and Effect Analysis. Reliability and validity of the instrument were evaluated using multidisciplinary simulation-based airway management training sessions. Each session was evaluated by 3 independent expert raters. A global assessment of the team performance and the previous experience in training were used to evaluate the validity of the instrument.
Results: The Just-in-Time Pediatric Airway Provider Performance Scale (JIT-PAPPS) version 3, with 34 task-based items (14 technical, 20 behavioral), was developed. Eighty-five teams led by resident airway providers were evaluated by 3 raters. The intraclass correlation coefficient for raters was 0.64. The JIT-PAPPS score correlated well with the global rating scale (r = 0.71, P < .001). Mean total scores across the teams were positively associated with resident previous training participation (β coefficient 7.1 ± 0.9, P < .001), suggesting good validity of the scale.
Conclusions: A task-based scoring instrument for a primary airway provider's performance with a multidisciplinary pediatric ICU team on simulated pediatric respiratory failure was developed. Reliability and validity evaluation supports the developed scale.