A prospective comparison of evidence synthesis search strategies developed with and without text-mining tools

Robin A Paynter; Robin Featherstone; Elizabeth Stoeger; Celia Fiordalisi; Christiane Voisin; Gaelen P Adam

doi:10.1016/j.jclinepi.2021.03.013

A prospective comparison of evidence synthesis search strategies developed with and without text-mining tools

J Clin Epidemiol. 2021 Nov:139:350-360. doi: 10.1016/j.jclinepi.2021.03.013. Epub 2021 Mar 20.

Authors

Robin A Paynter¹, Robin Featherstone², Elizabeth Stoeger², Celia Fiordalisi², Christiane Voisin², Gaelen P Adam²

Affiliations

¹ Scientific Resource Center, AHRQ Effective Health Care Program, Portland VA Research Foundation 3710 SW US Veterans Hospital Road, Portland, OR 97239, USA. Electronic address: robin.paynter@va.gov.
² Scientific Resource Center, AHRQ Effective Health Care Program, Portland VA Research Foundation 3710 SW US Veterans Hospital Road, Portland, OR 97239, USA.

PMID: 33753230
DOI: 10.1016/j.jclinepi.2021.03.013

Abstract

Objective: We compared the process of developing searches with and without using text-mining tools (TMTs) for evidence synthesis products.

Study design: This descriptive comparative analysis included seven systematic reviews, classified as simple or complex. Two librarians created MEDLINE strategies for each review, using either usual practice (UP) or TMTs. For each search we calculated sensitivity, number-needed-to-read (NNR) and time spent developing the search strategy.

Results: We found UP searches were more sensitive (UP 92% (95% CI, 85-99); TMT 84.9% (95% CI, 74.4-95.4)), with lower NNR (UP 83 (SD 34); TMT 90 (SD 68)). UP librarians spent an average of 12 h (SD 8) developing search strategies, compared to TMT librarians' 5 hours (SD 2).

Conclusion: Across all reviews, TMT searches were less sensitive than UP searches, but confidence intervals overlapped. For simple SR topics, TMT searches were faster and slightly less sensitive than UP. For complex SR topics, TMT searches were faster and less sensitive than UP searches but identified unique eligible citations not found by the UP searches.

Keywords: Comparative Studies; Evidence Synthesis; Information Retrieval; PubMed; Systematic review; Text-mining.

Publication types

Comparative Study

MeSH terms

Data Collection / standards*
Data Collection / statistics & numerical data*
Data Mining / standards*
Data Mining / statistics & numerical data
Databases, Bibliographic / standards*
Databases, Bibliographic / statistics & numerical data
Humans
Information Storage and Retrieval / standards*
Information Storage and Retrieval / statistics & numerical data*
MEDLINE / statistics & numerical data
Prospective Studies
Systematic Reviews as Topic / standards*