New Search Strategies Optimize MEDLINE Retrieval of Sound Studies on Treatment or Prevention of Health Disorders

Abstract

A review of:

Haynes, R. Brian, K. Ann McKibbon, Nancy L. Wilczynski, Stephen D. Walter, and Stephen R. Were. “Optimal search strategies for retrieving scientifically strong studies of treatment from Medline: analytical survey.” BMJ 330.7501 (21 May 2005): 1179.

Objective – To develop and test search strategies for retrieving clinically sound studies from the MEDLINE database on the prevention or treatment of health disorders.

Design – Analytical survey.

Subjects – The data sources were articles about treatment studies selected from 161 journal titles indexed for MEDLINE in the year 2000.

Setting – MEDLINE database searches performed at the Health Information Research Unit, McMaster University, in Ontario, Canada.

Methods – Researchers hand searched each issue of 161 journal titles indexed in MEDLINE in the year 2000 to find treatment studies. Journal content included internal medicine, family practice, nursing, and mental health titles. Selected studies met the following criteria: randomisation of subjects, outcome assessment for at least 80% of who entered the study, and an analysis consistent with study design. Of 49,028 potential articles, 6,568 were identified as being treatment or prevention related, and 1,587 met the evaluation criteria. The study authors then created search strategies designed to retrieve articles in MEDLINE that met the same criteria, while excluding articles that did not. They compiled a list of 4,862 unique terms related to study criteria, and tested them using the Ovid Technologies search platform. Overall, 18,404 multiple-term search strategies were tested. Single terms with specificity greater than 75% and sensitivity greater than 25% were combined into strategies with two or more terms. These multiple term strategies were tested if they yielded sensitivity or accuracy greater than 75% and specificity of at least 50%.

Main Results – Of the 4,862 unique terms, 3,807 retrieved citations from MEDLINE that researchers used to assess sensitivity, specificity, precision, and accuracy. The single term that yielded the best accuracy while keeping sensitivity greater than 50% was ‘randomized controlled trial.pt.’.The single term that yielded the best precision while keeping sensitivity greater than 50% was also ‘randomized controlled trial.pt.’.This term also gave the greatest balance of sensitivity and specificity. Combination strategies varied. Some two-term combinations outperformed single term strategies and three-term combinations. Tables in the article provide the top three search strategies yielding the highest sensitivity, specificity, accuracy, and balance between sensitivity and specificity.

Search strategies with sensitivity greater than 50% and specificity greater than 95% were evaluated further by adding search terms using logistic regression techniques. The best strategies for maximizing sensitivity had sensitivity greater than 99% and specificity higher than 70%. The best strategies for maximizing specificity had sensitivity greater than 93% and specificity more than 97%.

Conclusion – In addition to providing the best strategies developed in this study, authors compared their results with the results from 19 other published strategies. The published strategies had a sensitivity range of 1.3% to 98.8% on the basis of the hand searched articles. These were all lower than this study’s best sensitivity of 99.3%. Two strategies published by Dumbrigue, with specificities of 98.1% and 97.6%, outperformed this study’s most specific strategy of 97.4%.