The Multilevel Reliability and Interrater Agreement of the Nurse Practitioner Primary Care Organizational Climate Questionnaire

J Nurs Meas. 2023 Sep 1;31(3):448-457. doi: 10.1891/JNM-2021-0082. Epub 2023 Aug 9.

Abstract

Background and Purpose: It is critical to accurately measure and understand the nurse practitioner (NP) work environment in which individual NP information is gathered but decisions or inferences are made at an aggregated, group level. However, there is little research on preconditions of aggregating individual-level data into group level in nursing research. This study was conducted to assess the multilevel reliability and group interrater agreement (IRA) of the Nurse Practitioner Primary Care Organizational Climate Questionnaire. Methods: Secondary data analysis from a cross-sectional survey with 247 NPs in 112 practices across 6 U.S. states in 2018-2019 was used. Results: The generalizability coefficients and values of IRA were acceptable (> .70), and values of ICC(1) were significant (.18-.38). Conclusions: Aggregating individual NP data to a higher group-level work environment construct is acceptable.

Keywords: generalizability theory; interrater agreement; multilevel reliability; work environment.

Publication types

  • Research Support, N.I.H., Extramural

MeSH terms

  • Cross-Sectional Studies
  • Humans
  • Nurse Practitioners*
  • Primary Health Care*
  • Reproducibility of Results
  • Surveys and Questionnaires