Risk Profile May Affect Search Process but Not Results

Abstract

Objective – To compare the use, in terms of process and outcomes, of electronic information resources by primary care physicians with different risk profiles and comfort with uncertainty.

Design – Survey, and observation using “think-aloud” method.

Setting – Physicians’ offices.

Subjects – Canadian and U.S. primary care physicians who report seeing patients in clinic settings.

Methods – Volunteers were recruited from personal contacts and the list of physicians who rate current studies for the McMaster Online Rating of Evidence (MORE) project. Physicians completed the Pearson scale to measure attitude toward risk and the Gerrity scale to measure comfort with uncertainty, and those who scored at the extremes of each of these two scales were included in the study (n=25), resulting in four groups (risk-seeking, risk-avoiding, uncertainty-stressed, uncertainty-unstressed). One researcher observed each of these physicians in their offices for an hour during which they completed questionnaires about their computer skills and familiarity with resources, answered multiple-choice clinical questions, and indicated level of certainty with regard to those answers (scale of 0 to 100%). Physicians also chose two of the clinical questions to answer using their own resources. The think-aloud method was employed, and transcripts were coded and analyzed.

Main results – The study analysis included two comparisons: risk-seeking (11 subjects) versus risk-avoiding (11 subjects) physicians, and uncertainty-stressed (11 subjects) versus uncertainty-unstressed (10 subjects) physicians. Most physicians were included in both sets of analyses. The researchers found no association of risk attitude and uncertainty stress with computer skills nor with familiarity and use of specific information resources (Internet, MEDLINE, PIER, Clinical Evidence, and UpToDate). No differences were found for the following outcomes: time spent searching, answers correct before searching, answers correct after searching, and certainty of answer if answer is right, certainty of answer if answer is wrong. There was a statistically significant association of participants’ indicating certainty for answers that were correct versus those that were not correct (p<0.03). Across all groups, the percentage of correct answers after searching was low, with the uncertainty-stressed group achieving the highest at 55.5%. There were differences observed in search process, including a trend toward a statistically significant difference in the number and proportion of search methods used across groups between the risk-seeking versus the risk-avoiding groups. Risk-seekers and those not stressed by uncertainty used more search heuristics (rules of thumb or guides that work in most circumstances) than their counterparts when searching. Those stressed by uncertainty tended to use MEDLINE less than those not stressed by uncertainty.

Conclusion – Based on a small sample, it appears that primary care physicians who are risk-seeking and/or less stressed by uncertainty have the same (poor) search results as their more risk-averse and stressed-by-uncertainty colleagues although their search processes may differ.