Background: In meta-analyses (MA), effect estimates that are pooled together will often be heterogeneous. Determining how substantial heterogeneity is is an important aspect of MA.
Method: We consider how best to quantify heterogeneity in the context of individual participant data meta-analysis (IPD-MA) of binary data. Both two- and one-stage approaches are evaluated via simulation study. We consider conventional I 2 and R 2 statistics estimated via a two-stage approach and R 2 estimated via a one-stage approach. We propose a simulation-based intraclass correlation coefficient (ICC) adapted from Goldstein et al. to estimate the I 2, from the one-stage approach.
Results: Results show that when there is no effect modification, the estimated I 2 from the two-stage model is underestimated, while in the one-stage model, it is overestimated. In the presence of effect modification, the estimated I 2 from the one-stage model has better performance than that from the two-stage model when the prevalence of the outcome is high. The I 2 from the two-stage model is less sensitive to the strength of effect modification when the number of studies is large and prevalence is low.
Conclusions: The simulation-based I 2 based on a one-stage approach has better performance than the conventional I 2 based on a two-stage approach when there is strong effect modification with high prevalence.
Keywords: Heterogeneity; I 2; Individual participant data meta-analysis (IPD-MA); Two-stage and one-stage approaches.