Undergraduate Information Literacy Instruction Is Not Enough to Prepare Junior Doctors for Evidence Based Practice

Abstract

Objective – To determine if junior doctors entering the workforce retain the information literacy skills they learned as undergraduates.

Design – Structured interviews and observations.

Setting – Wellington Medical School of the University of Otago in New Zealand. Medicine in New Zealand is an undergraduate program.

Subjects – Thirty-eight University of Otago students who were starting their fourth year of undergraduate medical training between 1994 and 2004. At the time of this study, the students had graduated and were a number of years into advanced training for their speciality, i.e., junior doctors. The participants represented five cohorts, each having received a different level of information literacy instruction as undergraduates. Cohort 1, with the most years in clinical practice at the time of the study, received no formal information literacy instruction as undergraduates. Cohorts 2 to 5 received information literacy instruction in their fourth undergraduate year. The focus of instruction for cohorts 2 and 3 was on developing an effective search strategy, whereas the instruction for cohorts 4 and 5 focused more on the critical appraisal of articles.

Methods – In 2008 and 2009, the authors contacted cohort graduates. Two medical librarians from the Wellington Medical Library interviewed and observed participants to establish their level of information literacy. The librarians asked an initial six questions to determine how much participants remembered of their undergraduate information literacy instruction, how they search for clinical information, what databases they use, how they evaluate information, and if they have had any formal or informal information literacy instruction since graduating. For question seven, participants described a recent situation in which they searched for clinical information relating to a given patient. For question eight, participants rated their own skill level as “no skills”, “some skills”, or “highly skilled” on the following seven parameters: choosing a source to search, brainstorming search terms, using Boolean operators, using database limits, finding randomized controlled trials (RCTs) and systematic reviews, using “explode” and “focus”, and evaluating articles. For the last question, the librarians observed and rated participants as they conducted a search in real-time. The librarians rated participants’ performance on the same parameters as question eight using the same scale of “no skills”, “some skills”, or highly skilled”.

Main Results – Most participants said they remembered or at least vaguely remembered the information literacy instruction they received in their fourth year. The authors noted that most participants seemed to have expanded on the skills they originally learned as undergraduates. Participants reported using a variety of information sources such as PubMed and Cochrane but were often unsure about what constitutes a database. A large number indicated that they use Google to find information. Rarely had participants asked a medical librarian for help with online searching.

The authors define evaluation as either intrinsic (based on information contained in the article itself) or extrinsic (based on such criteria as a journal’s reputation or its publisher). All the cohorts, even cohorts 4 and 5 who received the most instruction on critical appraisal, relied more or equally on extrinsic factors than on intrinsic factors.

When asked if they had received further formal or informal information literacy instruction since their fourth undergraduate year, most participants in cohorts 1 and 2 said they had. Fewer participants in cohorts 3, 4, and 5 indicated they had received further instruction.

The participants on average rated themselves highest on using database limits and lowest on using “explode” and “focus”. The observers on average rated the participants highest on choosing a source to search and lowest on finding RCTs and systematic reviews as well as using “explode” and “focus”, which tied for the lowest rating. The observed searches on average were rated lower than the self-assessments on all but one parameter. None of the average scores for either the self-rating or the observer-rating approached “highly skilled”.

Conclusion – The authors concluded that the information literacy instruction the participants received as undergraduates did not prepare them adequately for evidence-based practice. Even though most participants said they remembered their undergraduate information literacy instruction, neither the average scores for the self-rating nor the observer-rating approached “highly skilled”. From that they could surmise that the attainment of information literacy should be a career-long learning process, beginning with undergraduate instruction and extending throughout one’s clinical practice.

The authors also found that the level of instruction cohorts received as undergraduates did not seem to correspond to their current ability. Cohort 1, who received no information literacy instruction as undergraduates, scored higher on average than cohorts 3 and 4 on the self-assessment and higher than cohorts 3, 4, and 5 on the observer assessment. Cohort 1 also used more evidence based sources than did cohort 4, who received the most training on evidence-based medicine.

Cohorts 1 and 2 reported the most postgraduate information literacy instruction, leading the authors to postulate that the further along one is in his medical career, the more important evidence based practice, and thus information literacy instruction, becomes. Even with additional instruction, however, the participants did not seem prepared for evidence-based practice. The authors concluded that information literacy instruction during postgraduate training and clinical practice—possibly giving the doctor’s specialty consideration when designing instruction—might be more important than undergraduate instruction. They also concluded that information literacy instruction might be more effective when its importance is emphasized by senior clinicians.