Evidence Summary
Undergraduate Information
Literacy Instruction Is Not Enough to Prepare Junior Doctors for Evidence Based
Practice
A Review of:
Cullen,
R., Clark, M., & Esson, R. (2011). Evidence-based
information-seeking skills of junior doctors entering the workforce: An
evaluation of the impact of information literacy training during pre-clinical
years. Health Information & Libraries
Journal, 28(2), 119-129. doi:10.1111/j.1471-1842.2011.00933.x
Reviewed by:
Carol
D. Howe
Reference
Librarian/Associate Professor
Gabriele
Library, Immaculata University
Immaculata,
Pennsylvania, United States of America
Email:
chowe@immaculata.edu
Received: 9
Feb. 2012 Accepted: 1 May 2012
2012 Howe.
This is an Open Access article distributed under the terms of the Creative
Commons‐Attribution‐Noncommercial‐Share Alike License 2.5 Canada (http://creativecommons.org/licenses/by-nc-sa/2.5/ca/),
which permits unrestricted use, distribution, and reproduction in any medium,
provided the original work is properly attributed, not used for commercial
purposes, and, if transformed, the resulting work is redistributed under the
same or similar license to this one.
Abstract
Objective – To
determine if junior doctors entering the workforce retain the information
literacy skills they learned as undergraduates.
Design – Structured
interviews and observations.
Setting – Wellington
Medical School of the University of Otago in New
Zealand. Medicine in New Zealand is an undergraduate program.
Subjects – Thirty-eight University of Otago
students who were starting their fourth year of undergraduate medical training
between 1994 and 2004. At the time of this study, the students had graduated
and were a number of years into advanced training for their speciality, i.e.,
junior doctors. The participants represented five cohorts, each having received
a different level of information literacy instruction as undergraduates. Cohort
1, with the most years in clinical practice at the time of the study, received
no formal information literacy instruction as undergraduates. Cohorts 2 to 5 received information literacy
instruction in their fourth undergraduate year. The focus of instruction for
cohorts 2 and 3 was on developing an effective search strategy, whereas the
instruction for cohorts 4 and 5 focused more on the critical appraisal of
articles.
Methods – In
2008 and 2009, the authors contacted cohort graduates. Two medical librarians
from the Wellington Medical Library interviewed and observed participants to
establish their level of information literacy. The librarians asked an initial
six questions to determine how much participants remembered of their
undergraduate information literacy instruction, how they search for clinical
information, what databases they use, how they evaluate information, and if
they have had any formal or informal information literacy instruction since
graduating. For question seven, participants described a recent situation in
which they searched for clinical information relating to a given patient. For
question eight, participants rated their own skill level as “no skills”, “some
skills”, or “highly skilled” on the following seven parameters: choosing a
source to search, brainstorming search terms, using Boolean operators, using
database limits, finding randomized
controlled trials (RCTs) and systematic reviews, using “explode” and “focus”,
and evaluating articles. For the last question, the librarians observed and
rated participants as they conducted a search in real-time. The librarians
rated participants’ performance on the same parameters as question eight using
the same scale of “no skills”, “some skills”, or highly skilled”.
Main Results –
Most participants said they remembered or
at least vaguely remembered the information literacy instruction they received
in their fourth year. The authors noted that most participants seemed to have
expanded on the skills they originally learned as undergraduates. Participants
reported using a variety of information sources such as PubMed and Cochrane but
were often unsure about what constitutes a database. A large number indicated
that they use Google to find information. Rarely had participants asked a medical
librarian for help with online searching.
The
authors define evaluation as either intrinsic (based on information contained
in the article itself) or extrinsic (based on such criteria as a journal’s
reputation or its publisher). All the cohorts, even cohorts 4 and 5 who
received the most instruction on critical appraisal, relied more or equally on
extrinsic factors than on intrinsic factors.
When
asked if they had received further formal or informal information literacy
instruction since their fourth undergraduate year, most participants in cohorts
1 and 2 said they had. Fewer participants in cohorts 3, 4, and 5 indicated they
had received further instruction.
The
participants on average rated themselves highest on using database limits and
lowest on using “explode” and “focus”. The observers on average rated the
participants highest on choosing a source to search and lowest on finding RCTs
and systematic reviews as well as using “explode” and “focus”, which tied for
the lowest rating. The observed searches on average were rated lower than the
self-assessments on all but one parameter. None of the average scores for
either the self-rating or the observer-rating approached “highly skilled”.
Conclusion – The
authors concluded that the information literacy instruction the participants
received as undergraduates did not prepare them adequately for evidence-based
practice. Even though most participants said they remembered their
undergraduate information literacy instruction, neither the average scores for
the self-rating nor the observer-rating approached “highly skilled”. From that
they could surmise that the attainment of information literacy should be a
career-long learning process, beginning with undergraduate instruction and
extending throughout one’s clinical practice.
The
authors also found that the level of instruction cohorts received as
undergraduates did not seem to correspond to their current ability. Cohort 1,
who received no information literacy instruction as undergraduates, scored
higher on average than cohorts 3 and 4 on the self-assessment and higher than
cohorts 3, 4, and 5 on the observer assessment. Cohort 1 also used more
evidence based sources than did cohort 4, who received the most training on
evidence-based medicine.
Cohorts
1 and 2 reported the most postgraduate information literacy instruction,
leading the authors to postulate that the further along one is in his medical
career, the more important evidence based practice, and thus information
literacy instruction, becomes. Even with additional instruction, however, the
participants did not seem prepared for evidence-based practice. The authors
concluded that information literacy instruction during postgraduate training
and clinical practice—possibly giving the doctor’s specialty consideration when
designing instruction—might be more important than undergraduate instruction.
They also concluded that information literacy instruction might be more
effective when its importance is emphasized by senior clinicians.
Commentary
The
ability to make well-informed clinical decisions is an indispensable skill for
doctors to have. The importance of this study is highlighted when one considers
how lack of training on how to practice evidence based medicine can affect
patient outcomes. By comparing the participants’ self-ratings with the
observers’ ratings, the authors found that the participants thought more highly
of their skills than was warranted. This could create a hazardous situation in
which doctors do not fully explore the body of evidence available to them.
The
more one uses a skill, the more one perfects it. This supports the notion that,
not just medical students, but also practicing doctors, should regularly
exercise their information literacy skills. Many participants indicated that
they had had further information literacy instruction since they were
undergraduates, which is promising. This might explain the authors’ observation
that most participants seemed to have expanded on the skills they were
originally taught as undergraduates and also the inconsistency in skill level
among cohorts.
The
validity of this study is weakened by several flaws. First, the sample size was
small, and all participants came from the same university. Secondly, the
ten-year time span between cohorts made comparison among them questionable,
especially considering that the Web was new in 1995 and commonplace by 2004.
Moreover, the content and search interfaces of the tools participants reported
using had probably changed quite a bit over ten years. The participants were
asked how much they remembered of their undergraduate information literacy
instruction, but they were asked anywhere from 4 to 14 years after the fact.
This introduced the potential for inaccurate recollections. Finally, there are
many discrepancies between data reported in the text of the article and its
tabular representation. For example, the Findings section states that 27
participants consulted a librarian ‘occasionally’ or ‘rarely’, whereas Table 3
indicates that only two participants asked a librarian for help.
Despite
its weaknesses, this study is important because it underscores the need for
future research of this type. It would be interesting to see results from
medical students in countries other than New Zealand, for example. Also, since
there is a growing emphasis on and new resources for evidence-based medicine,
it would be helpful to see the results from a more current study. Given the
authors’ finding that junior doctors are insufficiently prepared for evidence based
practice, further research is critical to highlight the importance of
career-long information literacy learning.