Google Scholar Retrieves Twice as Many Relevant Citations as PubMed and Provides Greater Full-Text Access for Quick, Clinical Nephrology Searches

Abstract

Objective – To compare recall and precision of results retrieved by searches in PubMed and Google Scholar for clinical nephrology literature.

Design – Survey questionnaire, comparative.

Setting – Canada.

Subjects – Practicing nephrologists with average age of 48 years and who have practiced nephrology for an average of 15 years.

Methods – The researchers identified 100 systematic reviews in renal therapy published between 2001 and 2009. The primary studies cited in the systematic reviews served as the reference standard for relevant articles; 1,574 unique citations were identified and used to measure recall and precision. The researchers created a unique clinical question from each of the objective statements of systematic reviews and sent one question to a random sample of practicing nephrologists to determine the search strings they would use to search for clinical literature; the researchers collected 100 usable responses. Using the search string in both Google Scholar and PubMed, the researchers analyzed the first 40 retrieved results in each for recall of relevant literature and precision. The researchers also analyzed the availability of full-text articles in each database. A pilot study to test the methodology preceded the main study.

Results – Google Scholar’s recall for the first 40 records was 21.9% and PubMed was 10.9%. Each database contained 78% of the relevant literature/reference standard set from the systematic reviews. However, 15% of the articles were in neither database. Precision results were similar (7.6% for Google Scholar and 5.6% for PubMed). Google Scholar had more full-text available at 15% of articles versus 5% for PubMed. Google Scholar and PubMed had similar numbers of relevant articles when all retrieved records were analyzed, but Google Scholar still provided more access to free full-text articles.

Conclusion – Google Scholar provides better recall and provides more access to full-text than PubMed; however, search strings provided by nephrologists used in both databases failed to retrieve 80% of relevant articles. Therefore improving nephrologists’ ability to effectively search could enhance their ability to implement research in practice helping patients. The researchers suggest future studies should be conducted to determine the generalizability of the findings on recall and precision in other medical disciplines.