Evaluation of the Long-Term Impact of a Curriculum-Integrated Medical Information Literacy Program





Information literacy, program evaluation, skills retention, medical students, undegraduate medical education


Introduction: Medical libraries have long provided educational programs to support evidence-based practice. Medical students at Queen’s University, Kingston, Ontario, participate in a curriculum-integrated information literacy program during the first two years of medical school. Do they retain, improve, or forget the skills? Do they continue to use the library resources during clerkship? Did they encounter barriers to prevent them from using the resources? 

Methods: A short survey was administered to 99 students at the end of medical school. The survey included questions about medical students’ attitudes and behaviours, their use of information resources, and their medical information literacy knowledge. Some of the knowledge questions were compared to pre- and post-tests that the same class completed in first year. Results: Fifty-three students completed the survey. The students rated their abilities very highly but there was only a weak positive relationship with the knowledge scores. Information resources were well used, both for clinical questions and to complete the mini-scholar exercises.

Discussion: Medical students feel better prepared to answer clinical questions and their skills improved or remained the same for the content that could be compared between first and fourth year. Different resources were used for day-to day information needs and for the completion of the minischolar exercises. The results will inform changes to the Medical Information Literacy program at Queen’s University. The librarians will explore some of the barriers to access to ensure that future students can use information resources with more ease while away from campus.

Author Biographies

Suzanne Maranda, Queen's University, Kingston, On.

Head of the Bracken Health Sciences Library since 2005.

Brittany Harding, Queen's University, Kingston, On.

Research Assistant, Office of Health Sciences Education

Laura Kinderman, Queen's University, Kingston, On.

Associate Director, Office of Health Sciences Education &
Associate Director, Bachelor of Health Sciences


(1) Maggio LA, Tannery NH, Chen HC, ten Cate O, O’Brien B. Evidence-based medicine training in undergraduate medical education: a review and critique of the literature published 20062011. Acad Med. 2013;88(7):10221028. PubMed PMID: 23702528. doi:10.1097/ACM.0b013e3182951959

(2) Shaneyfelt T, Baum K, Bell D, Feldstein T, Kaatz S, Whelan C, et al., Instruments for evaluating education in evidence-based practice: a systematic review. JAMA. 2006;296:1116-1127. doi:10.1001/jama.296.9.1116

(3) Just ML. Is literature search training for medical students and residents effective? A literature review. J Med Libr Assoc. 2012;100(4):270276. PubMed PMID: 23133326; PubMed Central PMCID: PMCPMC3484946.

(4) Gruppen LD, Rana GK, Arndt TS. A controlled comparison study of the efficacy of training medical students in evidence-based medicine literature searching skills. Acad Med. 2005; 80(10):940944. PubMed PMID: 00001888-200510000-00014.

(5) Dorsch JL, Aiyer MK, Gumidyala K, Meyer LE. Retention of EBM competencies. Med Ref Serv Q. 2006;25(3):4557. doi:10.1300/J115v25n03_04

(6) Green ML, Ruff TR. Why do residents fail to answer their clinical questions? A qualitative study of barriers to practicing evidence-based medicine. Acad Med. 2005;80(2): 176182. PubMed PMID: 15671325. doi:10.1097/00001888-200502000-00016

(7) Cullen R, Clark M, Esson R. Evidence-based information seeking skills of junior doctors entering the workforce: an evaluation of the impact of information literacy training during pre-clinical years. Health Info Libr J. 2011;28(2):119129. doi:10.1111/j.1471-1842.2011.00933.x

(8) Stevenson P. Evaluating educational interventions for information literacy. Health Info Libr J. 2012;29(1):8186. doi:10.1111/j.1471-1842.2011.00976.x

(9) McKibbon KA, Fridsma DB. Effectiveness of clinician selected electronic information resources for answering primary care physicians’ information needs. J Am Med Inform Assoc. 2006;13(6):653659. PubMed PMID: PMC1656967. doi:10.1197/jamia.M2087

(10) West CP, Jaeger TM, McDonald FS. Extended evaluation of a longitudinal medical school evidence-based medicine curriculum. J Gen Intern Med. 2011;26(6):611615. PubMed PMID: 21286836; PubMed Central PMCID: PMCPMC 3101983.

(11) Ramos K, Schafer S, Tracz S. Validation of the Fresno test of competence in evidence based medicine. BMJ. 2003;326: 319321. doi:10.1136/bmj.326.7384.319

(12) Salbach NMPTP, Jaglal SBP, Williams JIP. Reliability and validity of the Evidence-Based Practice Confidence (EPIC) scale. J Contin Educ Health Prof. 2013;33(1):3340. doi: 10.1002/chp.21164

(13) Ilic D, Nordin RB, Glasziou P, Tilson JK, Villanueva E. Development and validation of the ACE tool: assessing medical trainees’ competency in evidence based medicine. BMC Med Educ. 2014;14(1):16. doi:10.1186/1472-6920-14-114

(14) Sanfilippo A. Undergraduate School of Medicine Blog [Internet]. Sanfilippo A, editor. Kingston, ON: School of Medicine, Queen’s University; 2016 [cited 4 Jul 2016]. Available from: http://meds.queensu.ca/blog/undergraduate/?p2986

(15) Kruger J, Dunning D. Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. J Pers Soc Psychol. 1999;77(6): 11211134. doi:10.1037/0022-3514.77.6.1121

(16) Pickett KM. Resource format preferences across the medical curriculum. J Med Libr Assoc. 2016;104(3):193196. PubMed PMID: 27366119; PubMed Central PMCID: PMCPMC4915636. doi:10.3163/1536-5050.104.3.003

(17) Shanahan MC. Transforming information search and evaluation practices of undergraduate students. Intern J Med Inform. 2008;77(8):518526. PubMed PMID: 18032097. doi:10.1016/j.ijmedinf.2007.10.004

(18) Younger P. Internet-based information-seeking behaviour amongst doctors and nurses: a short review of the literature. Health Inform Libr J. 2010;27(1):210. PubMed PMID: 20402799. doi:10.1111/j.1471-1842.2010.00883.x



How to Cite

Maranda, S., Harding, B., & Kinderman, L. (2016). Evaluation of the Long-Term Impact of a Curriculum-Integrated Medical Information Literacy Program. Journal of the Canadian Health Libraries Association Journal De l’Association Des bibliothèques De La Santé Du Canada, 37(3). https://doi.org/10.5596/c16-026



Research Articles