Perceived information gain from randomized trials correlates with publication in high-impact factor journals

Abstract

Objective: To examine whether perceived information gain (IG) drives the publication of randomized trials in high-impact factor (IF) journals.

Study design and setting: We estimated IG as the Kullback-Leibler divergence, quantifying how much a new finding changes established knowledge. We used 67 meta-analyses (964 randomized trials) that include one or more trials from any of the three highest IF general medical journals (NEJM, JAMA, and Lancet). We calculated IG for the presence of a non-null effect (IG(1)) and IG for the effect size magnitude (IG(2)).

Results: Across meta-analyses, the summary correlation coefficient of IF was 0.23 (95% confidence interval [CI]: 0.14, 0.31) for IG(1) and 0.35 (95% CI: 0.25, 0.46) for IG(2). IF also correlated with the P-value of the results (r=0.18), order of publication (r=-0.13), and number of events in the trial (r=0.36). Multivariate regression including IG, order of publication, P-value, and the number of events showed that IG is an independent correlate of IF. IG(2) explained a substantially larger proportion of the variance in IF than IG(1).

Conclusion: Publication in journals with high IF is driven by how extensively the results of a study change prior perceptions of the evidence, independently of the statistical significance and size of the study.