For Non-expert Clinical Searches, Google Scholar Results are Older with Higher Impact while PubMed Results Offer More Breadth

Abstract

Objectives – To compare PubMed and Google Scholar results for content relevance and article quality

Design – Bibliometric study.

Setting – Department of Internal Medicine at Texas Tech University Health Sciences Center.

Methods – Four clinical searches were conducted in both PubMed and Google Scholar. Search methods were described as “real world” (p. 216) behaviour, with the searchers familiar with content, though not expert at retrieval techniques. The first 20 results from each search were evaluated for relevance to the initial question, as well as for quality.

Relevance was determined based on one author’s subjective assessment of information in the title and abstract, when available, and then tested by two other authors, with discrepancies discussed and resolved. Items were assigned to one of three categories: relevant, possibly relevant, and not relevant to the question, with reviewer agreement measured using a weighted kappa statistic. The quality of items found to be ‘relevant’ and ‘possibly relevant’ was measured by impact factor ratings from Thomsen Reuters (ISI) Web of Knowledge, when available, as well as information obtained by SCOPUS on the number of times items were cited.

Main Results – Google Scholar results were judged to be more relevant and of higher quality than results obtained from PubMEed. Google Scholar results are also older on average, while PubMed retrieved items from a larger number of unique journals.

Conclusion – In agreement with earlier research, the authors recommended that searchers use both PubMed and Google Scholar to improve on the quality and relevance of results. Searches in the two resources identify unique items based upon the ranking algorithms involved.