Residents and Medical Students Correctly Answer Clinical Questions More Often with Google and UpToDate than With PubMed or Ovid MEDLINE

Abstract

Objective – To determine which search tool (Google, UpToDate, PubMed or Ovid-MEDLINE) produces more accurate answers for residents, medical students, and attending physicians searching on clinical questions in anesthesiology and critical care. Searcher confidence in the answers and speed with which answers were found were also examined.

Design – Randomized study without a control group.

Setting – Large university medical center.

Subjects –Subjects included 15 fourth year medical students (third and fourth year), 35 residents, and 4 attending physicians volunteered and completed the study. One additional attending withdrew halfway through the study. The authors were unsuccessful in recruiting an equal number of subjects from each group.

Methods – A set of eight anesthesia and critical care questions was developed, based on their commonality and importance in clinical practice and their answerability. Four search tools were employed: Google, UpToDate, PubMed, and Ovid MEDLINE. In part I, subjects were given a random set of four of the questions to answer with the search tool(s) of their choice, but could use only one search tool per question. In part II, several weeks later, the same subjects were randomly assigned a search tool with which to answer all 8 questions. The authors state that “for data analysis, PubMed was arbitrarily chosen to be the “reference standard.”” Statistical analysis was used to identify significant differences between PubMed and the other search tools.

Main Results – Part I: Subjects choosing a search tool were more likely to find a correct answer with Google or UpToDate. There were no statistically significant differences in confidence with answers between any of the search tools and PubMed.

Part II: Though subjects were assigned a search tool, some questions were repeated from part I. For repeated questions, Ovid users (compared to PubMed users) were significantly less likely to find the correct answer for repeated questions. Otherwise, there was no statistically significant difference in questions answered correctly. Confidence did not differ. When asked to answer new questions, subjects using Google and UpToDate were significantly more likely to find a correct answer than PubMed users. UpToDate users were more confident. There was no statistical difference in primary outcome (correct answer with high confidence) between Google, Ovid, and PubMed.

Pooled data from parts I and II, removing repeated questions: Subjects using Google and UpToDate were more likely to find correct answers. Confidence was highest among UpToDate users. Average search time per question (limited to 5 minutes per question) in ascending order of time spent was: UpToDate, Google, PubMed, and Ovid.

Conclusion – While the number of participants is small, the results suggest that the popular search engine Google and the commercially produced secondary online source UpToDate are more useful and efficient for finding answers to questions arising in anesthesiology and critical care practice than tools focused exclusively on indexing the primary literature.