Evidence Summary

Undergraduate Information Literacy Instruction Is Not Enough to Prepare Junior Doctors for Evidence Based Practice

A Review of:

Cullen, R., Clark, M., & Esson, R. (2011). Evidence-based information-seeking skills of junior doctors entering the workforce: An evaluation of the impact of information literacy training during pre-clinical years. Health Information & Libraries Journal, 28(2), 119-129. doi:10.1111/j.1471-1842.2011.00933.x

Reviewed by:

Carol D. Howe

Reference Librarian/Associate Professor

Gabriele Library, Immaculata University

Immaculata, Pennsylvania, United States of America

Email: chowe@immaculata.edu

Received: 9 Feb. 2012 Accepted: 1 May 2012

2012 Howe. This is an Open Access article distributed under the terms of the Creative Commons‐Attribution‐Noncommercial‐Share Alike License 2.5 Canada (http://creativecommons.org/licenses/by-nc-sa/2.5/ca/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one.

Abstract

Objective – To determine if junior doctors entering the workforce retain the information literacy skills they learned as undergraduates.

Design – Structured interviews and observations.

Setting – Wellington Medical School of the University of Otago in New Zealand. Medicine in New Zealand is an undergraduate program.

Subjects – Thirty-eight University of Otago students who were starting their fourth year of undergraduate medical training between 1994 and 2004. At the time of this study, the students had graduated and were a number of years into advanced training for their speciality, i.e., junior doctors. The participants represented five cohorts, each having received a different level of information literacy instruction as undergraduates. Cohort 1, with the most years in clinical practice at the time of the study, received no formal information literacy instruction as undergraduates. Cohorts 2 to 5 received information literacy instruction in their fourth undergraduate year. The focus of instruction for cohorts 2 and 3 was on developing an effective search strategy, whereas the instruction for cohorts 4 and 5 focused more on the critical appraisal of articles.

Methods – In 2008 and 2009, the authors contacted cohort graduates. Two medical librarians from the Wellington Medical Library interviewed and observed participants to establish their level of information literacy. The librarians asked an initial six questions to determine how much participants remembered of their undergraduate information literacy instruction, how they search for clinical information, what databases they use, how they evaluate information, and if they have had any formal or informal information literacy instruction since graduating. For question seven, participants described a recent situation in which they searched for clinical information relating to a given patient. For question eight, participants rated their own skill level as “no skills”, “some skills”, or “highly skilled” on the following seven parameters: choosing a source to search, brainstorming search terms, using Boolean operators, using database limits, finding randomized controlled trials (RCTs) and systematic reviews, using “explode” and “focus”, and evaluating articles. For the last question, the librarians observed and rated participants as they conducted a search in real-time. The librarians rated participants’ performance on the same parameters as question eight using the same scale of “no skills”, “some skills”, or highly skilled”.

Main Results – Most participants said they remembered or at least vaguely remembered the information literacy instruction they received in their fourth year. The authors noted that most participants seemed to have expanded on the skills they originally learned as undergraduates. Participants reported using a variety of information sources such as PubMed and Cochrane but were often unsure about what constitutes a database. A large number indicated that they use Google to find information. Rarely had participants asked a medical librarian for help with online searching.

The authors define evaluation as either intrinsic (based on information contained in the article itself) or extrinsic (based on such criteria as a journal’s reputation or its publisher). All the cohorts, even cohorts 4 and 5 who received the most instruction on critical appraisal, relied more or equally on extrinsic factors than on intrinsic factors.

When asked if they had received further formal or informal information literacy instruction since their fourth undergraduate year, most participants in cohorts 1 and 2 said they had. Fewer participants in cohorts 3, 4, and 5 indicated they had received further instruction.

The participants on average rated themselves highest on using database limits and lowest on using “explode” and “focus”. The observers on average rated the participants highest on choosing a source to search and lowest on finding RCTs and systematic reviews as well as using “explode” and “focus”, which tied for the lowest rating. The observed searches on average were rated lower than the self-assessments on all but one parameter. None of the average scores for either the self-rating or the observer-rating approached “highly skilled”.

Conclusion – The authors concluded that the information literacy instruction the participants received as undergraduates did not prepare them adequately for evidence-based practice. Even though most participants said they remembered their undergraduate information literacy instruction, neither the average scores for the self-rating nor the observer-rating approached “highly skilled”. From that they could surmise that the attainment of information literacy should be a career-long learning process, beginning with undergraduate instruction and extending throughout one’s clinical practice.

The authors also found that the level of instruction cohorts received as undergraduates did not seem to correspond to their current ability. Cohort 1, who received no information literacy instruction as undergraduates, scored higher on average than cohorts 3 and 4 on the self-assessment and higher than cohorts 3, 4, and 5 on the observer assessment. Cohort 1 also used more evidence based sources than did cohort 4, who received the most training on evidence-based medicine.

Cohorts 1 and 2 reported the most postgraduate information literacy instruction, leading the authors to postulate that the further along one is in his medical career, the more important evidence based practice, and thus information literacy instruction, becomes. Even with additional instruction, however, the participants did not seem prepared for evidence-based practice. The authors concluded that information literacy instruction during postgraduate training and clinical practice—possibly giving the doctor’s specialty consideration when designing instruction—might be more important than undergraduate instruction. They also concluded that information literacy instruction might be more effective when its importance is emphasized by senior clinicians.

Commentary

The ability to make well-informed clinical decisions is an indispensable skill for doctors to have. The importance of this study is highlighted when one considers how lack of training on how to practice evidence based medicine can affect patient outcomes. By comparing the participants’ self-ratings with the observers’ ratings, the authors found that the participants thought more highly of their skills than was warranted. This could create a hazardous situation in which doctors do not fully explore the body of evidence available to them.

The more one uses a skill, the more one perfects it. This supports the notion that, not just medical students, but also practicing doctors, should regularly exercise their information literacy skills. Many participants indicated that they had had further information literacy instruction since they were undergraduates, which is promising. This might explain the authors’ observation that most participants seemed to have expanded on the skills they were originally taught as undergraduates and also the inconsistency in skill level among cohorts.

The validity of this study is weakened by several flaws. First, the sample size was small, and all participants came from the same university. Secondly, the ten-year time span between cohorts made comparison among them questionable, especially considering that the Web was new in 1995 and commonplace by 2004. Moreover, the content and search interfaces of the tools participants reported using had probably changed quite a bit over ten years. The participants were asked how much they remembered of their undergraduate information literacy instruction, but they were asked anywhere from 4 to 14 years after the fact. This introduced the potential for inaccurate recollections. Finally, there are many discrepancies between data reported in the text of the article and its tabular representation. For example, the Findings section states that 27 participants consulted a librarian ‘occasionally’ or ‘rarely’, whereas Table 3 indicates that only two participants asked a librarian for help.

Despite its weaknesses, this study is important because it underscores the need for future research of this type. It would be interesting to see results from medical students in countries other than New Zealand, for example. Also, since there is a growing emphasis on and new resources for evidence-based medicine, it would be helpful to see the results from a more current study. Given the authors’ finding that junior doctors are insufficiently prepared for evidence based practice, further research is critical to highlight the importance of career-long information literacy learning.