Multi-method process maps: An interdisciplinary approach to investigate ad hoc modifications in protocol-driven interventions

Thomas I Mackie; Leah Ramella; Ana J Schaefer; Monica Sridhar; Alice S Carter; Abbey Eisenhower; Grace T Ibitamuno; Marisa Petruccelli; Shawna V Hudson; R Christopher Sheldrick

doi:10.1017/cts.2020.14

Multi-method process maps: An interdisciplinary approach to investigate ad hoc modifications in protocol-driven interventions

J Clin Transl Sci. 2020 Feb 26;4(3):260-269. doi: 10.1017/cts.2020.14.

Authors

Affiliations

¹ Rutgers School of Public Health, Piscataway, NJ, USA.
² Institute for Health, Health Care Policy and Aging Research, Rutgers University, New Brunswick, NJ, USA.
³ Department of Health Law, Policy and Management, Boston University School of Public Health, Boston, MA, USA.
⁴ Rutgers Robert Wood Johnson Medical School, New Brunswick, NJ, USA.
⁵ Department of Clinical Psychology, University of Massachusetts, Boston, MA, USA.
⁶ Department of Family Medicine, UMDNJ-Robert Wood Johnson Medical School, New Brunswick, NJ, USA.

Abstract

Introduction: Implementation scientists increasingly recognize that the process of implementation is dynamic, leading to ad hoc modifications that may challenge fidelity in protocol-driven interventions. However, limited attention to ad hoc modifications impairs investigators' ability to develop evidence-based hypotheses about how such modifications may impact intervention effectiveness and cost. We propose a multi-method process map methodology to facilitate the systematic data collection necessary to characterize ad hoc modifications that may impact primary intervention outcomes.

Methods: We employ process maps (drawn from systems science), as well as focus groups and semi-structured interviews (drawn from social sciences) to investigate ad hoc modifications. Focus groups are conducted with the protocol's developers and/or planners (the implementation team) to characterize the protocol "as envisioned," while interviews conducted with frontline administrators characterize the process "as realized in practice." Process maps with both samples are used to identify when modifications occurred across a protocol-driven intervention. A case study investigating a multistage screening protocol for autism spectrum disorders (ASD) is presented to illustrate application and utility of the multi-method process maps.

Results: In this case study, frontline administrators reported ad hoc modifications that potentially influenced the primary study outcome (e.g., time to ASD diagnosis). Ad hoc modifications occurred to accommodate (1) whether providers and/or parents were concerned about ASD, (2) perceptions of parental readiness to discuss ASD, and (3) perceptions of family service delivery needs and priorities.

Conclusion: Investigation of ad hoc modifications on primary outcomes offers new opportunities to develop empirically based adaptive interventions. Routine reporting standards are critical to provide full transparency when studying ad hoc modifications.

Keywords: Implementation science; adaptation; modification; process assessment.

Abstract

Grants and funding