An Observed Structured Teaching Evaluation Demonstrates the Impact of a Resident-as-Teacher Curriculum on Teaching Competency

Matthew Zackoff; Karen Jerardi; Ndidi Unaka; Heidi Sucharew; Melissa Klein

doi:10.1542/hpeds.2014-0134

An Observed Structured Teaching Evaluation Demonstrates the Impact of a Resident-as-Teacher Curriculum on Teaching Competency

Hosp Pediatr. 2015 Jun;5(6):342-7. doi: 10.1542/hpeds.2014-0134.

Authors

Matthew Zackoff¹, Karen Jerardi², Ndidi Unaka², Heidi Sucharew², Melissa Klein²

Affiliations

¹ Department of Pediatrics, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio Matthew.zackoff@cchmc.org.
² Department of Pediatrics, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio.

PMID: 26034166
DOI: 10.1542/hpeds.2014-0134

Abstract

Background and objective: Residents play a critical role in the education of peers and medical students, yet attainment of teaching skills is not routinely assessed. The primary aim of this study was to develop a novel, skill-based Observed Structured Teaching Evaluation (OSTE) and self-assessment survey to measure the impact of a resident-as-teacher curriculum on teaching competency. The secondary aim was to determine interrater reliability of the OSTE.

Methods: A prospective study quantitatively assessed intern teaching competency via videotaped teaching encounters (videos) before and after a month-long hospital medicine rotation and self-assessment surveys over a 5-month period. The intervention group received the resident-as-teacher curriculum. Videos were evaluated by 2 blinded faculty via an OSTE covering 9 skills within 3 core components: preparation, teaching, and reflection. Pre- to post-HM rotation month differences were evaluated within and between groups using the Wilcoxon signed rank test and Wilcoxon rank-sum test, respectively.

Results: Twenty-two of 25 (88%) control and 27 of 28 (96%) intervention interns participated; 100% of participants completed the study. The intervention group's pre-post difference for the total OSTE score and the average self-assessed competence statistically improved; however, no significant difference was seen between groups. The difference in preparation scores was significant for the intervention compared with the control. The OSTE's interrater reliability demonstrated good agreement with weighted kappas of 0.86 for preparation, 0.71 for teaching, and 0.93 for reflection.

Conclusions: Implementation of an objective, skill-based OSTE detected observable changes in interns' teaching competency after implementation of a brief resident-as-teacher curriculum. The OSTE's good interrater reliability may allow standardized assessment of skill attainment over time.

Keywords: medical education; observed structured teaching evaluation (OSTE); resident-as-teacher.

Publication types

Research Support, Non-U.S. Gov't

MeSH terms

Adult
Curriculum*
Faculty / standards*
Female
Humans
Internship and Residency / standards*
Male
Professional Competence / standards*
Prospective Studies
Videotape Recording