Bringing the Scholar Competency to Life: Learning and Assessment 1

University students may be perceived as ‘‘digital natives’’, but their ability to effectively access valid information is often overestimated. Current research shows that they overuse Google, rarely evaluate resources, and encounter significant challenges during the research process [1, 2]. How can librarians and medical faculty encourage medical students to develop an efficient approach to accessing and evaluating information? While it is certainly an important part of the research process, addressing specific searching in medical databases should not be the first step. Even though some students arrive in medical school with Masters and Doctoral degrees, they are not necessarily familiar with the resources to help answer their background questions; this is the information they need to start building their medical knowledge base. Often the students’ first action is to search for information in Google when faced with an unknown word or disease. Medical libraries purchase a range of information resources with high credibility and that are supported by critically appraised evidence; librarians are consistently trying to improve access to these often expensive materials. Moreover, first-year medical students need to build their depth of knowledge in medicine. Connecting students with these resources has been a challenge at Queen’s University Library and led to the development of our blended learning curriculum for information literacy sessions [3]. At the same time, the Queen’s University Medical School was planning a curriculum redesign around competencies derived from the Roles of a Physician from the CanMEDS Framework [4]. These roles describe areas of competency required by effective physicians, but are skills separate from knowledge of disease manifestation and treatment. They include Collaborator, Communicator, Advocate, Manager, Professional, Scholar, as well as the Medical Expert. The research process is fundamental to the scholar role, and specific objectives related to competencies are now developed within the Medical Council of Canada’s (MCC) Objectives for the Qualifying Examination [5]. In the preclerkship curriculum we have refined these competencies to include: the ability to contribute to the process of knowledge creation (research), the capacity to engage in self-directed learning, and the ability to critically evaluate information and its sources (the literature). Our goal is to integrate these competencies with those of the Medical Expert role using the MCC’s patient presentations [6]. Central to the role of Scholar in Undergraduate Medicine is the formulation of effective questions and the evaluation of information resources using objective parameters, followed by the efficient searching of the chosen resource(s), the evaluation of the results, and critical appraisal of the retrieved content all culminating with the integration of new knowledge into practice. This focus on the roles and competencies laid out in CanMEDS created a perfect opportunity to work with a multidisciplinary team to further integrate information literacy into the redesigned medical curriculum.


Introduction
University students may be perceived as ''digital natives'', but their ability to effectively access valid information is often overestimated. Current research shows that they overuse Google, rarely evaluate resources, and encounter significant challenges during the research process [1,2]. How can librarians and medical faculty encourage medical students to develop an efficient approach to accessing and evaluating information?
While it is certainly an important part of the research process, addressing specific searching in medical databases should not be the first step. Even though some students arrive in medical school with Masters and Doctoral degrees, they are not necessarily familiar with the resources to help answer their background questions; this is the information they need to start building their medical knowledge base.
Often the students' first action is to search for information in Google when faced with an unknown word or disease. Medical libraries purchase a range of information resources with high credibility and that are supported by critically appraised evidence; librarians are consistently trying to improve access to these often expensive materials. Moreover, first-year medical students need to build their depth of knowledge in medicine. Connecting students with these resources has been a challenge at Queen's University Library and led to the development of our blended learning curriculum for information literacy sessions [3].
At the same time, the Queen's University Medical School was planning a curriculum redesign around competencies derived from the Roles of a Physician from the CanMEDS Framework [4]. These roles describe areas of competency required by effective physicians, but are skills separate from knowledge of disease manifestation and treatment. They include Collaborator, Communicator, Advocate, Manager, Professional, Scholar, as well as the Medical Expert. The research process is fundamental to the scholar role, and specific objectives related to competencies are now developed within the Medical Council of Canada's (MCC) Objectives for the Qualifying Examination [5]. In the preclerkship curriculum we have refined these competencies to include: the ability to contribute to the process of knowledge creation (research), the capacity to engage in self-directed learning, and the ability to critically evaluate information and its sources (the literature). Our goal is to integrate these competencies with those of the Medical Expert role using the MCC's patient presentations [6].
Central to the role of Scholar in Undergraduate Medicine is the formulation of effective questions and the evaluation of information resources using objective parameters, followed by the efficient searching of the chosen resource(s), the evaluation of the results, and critical appraisal of the retrieved content Á all culminating with the integration of new knowledge into practice. This focus on the roles and competencies laid out in CanMEDS created a perfect opportunity to work with a multidisciplinary team to further integrate information literacy into the redesigned medical curriculum.

Description
During the medical curriculum redesign at Queen's University, the teaching of information literacy was merged with the teaching of critical appraisal into a new course encompassing all of the steps in evidence-based medicine (EBM) [7]. This new course is delivered in the very first term of medical school, so that students will begin to develop their EBM skills as soon as they begin their medical learning. In the fall of 2010 this new Medical School course, Critical Appraisal, Research and Learning (CARL), was introduced with an information literacy component that required the medical students to complete two online tutorials paired with a follow-up library session. The two online tutorials were ''Dr. Google & e-Resources'' and ''Evidence: Search & Rescue''. These learning activities were built into the curriculum to prepare the students for additional CARL course requirements. Scaffolding the medical information literacy content with clinical casebased learning demonstrated the depth and types of authoritative information available to the students beyond Google. In addition to providing the opportunity for students to meet the learning objectives of the Scholar competency, the rationale for this blended learning approach includes: (i) Demonstrating the relevance of medical information skills. The cases in the modules offered a real-world application with progressive interaction between the student and the virtual patient, culminating in a differential diagnosis. Each step required the student to access relevant information using linked online library resources. This guided, practical tour of the resources allowed students to experience both the navigation process and the useful linkage of searching and acquiring practical information. The case-based framework was used similarly for each different skill or resource presented. Interactive formative and summative assessments linked to these tasks were included in the online modules [8Á12]. (ii) Increasing the awareness of reliable sources of medical information. The online modules presented authoritative and current e-resources (e.g., e-books, pointof-care tools, Medline database, etc.), or information beyond the discovery style searching Google provides, that students could use to develop their depth of knowledge in medicine. Through repeated exposure within the course framework, the students developed an enhanced awareness of the e-resources available to them. (iii) Facilitating self-directed and active learning. Students completed online modules independently, allowing students with varied backgrounds in literature searching to work in a self-regulated fashion, at their own pace, and complete online quizzes to self-assess learning [13,14]. After completion of the online modules, students attended small group sessions in the library. This format allowed for student practice and application of skills. In keeping with the blended learning approach, librarians built on tutorial content by facilitating collaborative learning with peer discussions [15Á18]. (iv) Creating a model to develop additional skills. The second blended learning session, Evidence: Search and Rescue, presented a point-of-care tool, the Patient, Intervention, Comparison, Outcome (PICO) question format and a Medline search for randomized controlled trial articles (RCTs). Again, this module was embedded within a clinical case that required the students to access additional information to complete the case. The EBM pyramid [19] was used in the follow-up session as an approach to the selection of information resources to solve clinical cases. Part of the summative assessment for the CARL course required the students to reproduce this process within a written assignment and to follow the EBM pyramid model to demonstrate their newly acquired research skills. (v) Ensuring consistency with continued ease of access, modularity and reusability. The tutorials were available online via the students' learning management system. They may be consulted by students for searching skills refinement during the remainder of their undergraduate medical school training. The modules are specifically referenced within other courses later in their training [16,17,20,21].

Development and description of the directed independent learning tutorials
Two web-based tutorials were created using open source software called eXe Learning [22], a standard format already in use at the medical school. The online tutorial content was developed by the CARL course multidisciplinary team interested in exploring a blended learning teaching methodology: the tutorials would be followed by in-class discussions and additional content to build on the knowledge gained in the tutorial. The focus was placed on medical information skills anchored in a clinical case designed to meet both the scholar competencies and selected MCC Medical Expert presentations [6].
The online tutorials were structured using a combination of clinical scenarios and screen shots of the library web pages, including e-books, point-of-care tools, and the Medline database. Each tutorial contained a variety of specific tasks nested in the material, designed to reinforce the process of accessing e-books, review articles, and literature searching in OvidSP Medline all while seeking information to assist the virtual patients.
Consistency in the format was provided by having the same structure for each topic in the tutorial, with specific tasks denoted by standardized icons. The tutorials were designed as independent steps in a progression from learning the structure and content of the online resources to basic, and then more advanced, search strategies for clinical queries.
Some common design features included: a home screen containing the course learning objectives and the interactive Table of Contents to provide quick navigation within the tutorial; tasks identified by icons, for students to practice skills and bring results to the follow-up small group learning sessions, (see Figure 1 for a sample screen); embedded comments from the physician educator to link the learning to medical practice information needs; criteria for use (e.g., currency, relevance, authority, accuracy, and purpose) when evaluating web sites [23]; case findings and conclusions to clinically evaluate the virtual patient's symptoms, diagnosis, and treatment; a summary section with key points to orient students to the learning outcomes of the module; and a formative, online, interactive quiz with immediate, automatic assessment of the clinical knowledge learned.

New resources
It was really valuable to learn how to use Ovid Medline and the clinical references (ACP-PIER, Five Minute Guide, etc.). I had never used any of those before and now I feel like I will be able to in the future Excellent to know about e-books! I think the resources this tutorial introduced us to were really interesting and will be so helpful in the future. These are REALLY good tools and I really enjoyed learning about them and doing them during the session. I learned a lot! I'm better aware of my resources now. Refworks was a very useful addition, will come in handy.
Blended Learning I found the Ovid search strategy and RefWorks tutorial most helpful. I think that this session was a useful introduction to the Queen's library resources. I have a graduate degree and I'm used to researching online resources but appreciated learning the OVID databases and how to integrate that to Refworks. The most helpful part was the lyme disease Medline search explanation and how to use e-books! Great clinical twist on utilizing online resources. Very well designed workshop. I like that we used a specific example (lyme disease) to practice the skills for using the difference resources. Very useful. Before the session, I considered myself fairly proficient with regards to primary article searching but I have now realized that I can be much more efficient. I thought the group session and having to do the cases was a great idea. I have previous experience with Medline, and so feel I developed knowledge of it. However, I do enjoy the activities in the tutorials, and they force me to keep current on my searching skills. Very informative, interesting, and fun.

Outcomes
Each year students were given the opportunity to evaluate the online modules. The positive feedback reassured the authors that the objectives were met: the students learned about new resources and they found this blended learning approach to be effective. Table 1 shows a sample of student comments collected after the completion of each in-class session.
The major CARL course assignment was worth 15% of the course mark and included a section, worth 5% of the total course grade, to assess the students' information literacy skills. This section required students to define their topic using background information resources, then prepare a research question using the PICO format [24], and perform a Medline search for a reliable RCT that would answer their question.
The librarians marked the information literacy section and in 2010 and 2011 the students performed very well with class averages of 4.1/5 and 4.3/5, respectively. Over both years, the marks ranged from 2.5/5 to 5/5. In 2010, 16% of the class achieved a perfect mark for this work, while in 2011 it was 25% of the class ( Table 2). The 2012 results had not been compiled at the time of writing.
At the end of the second library session, the students are asked to complete a short quiz that is part of a thread of 10 CARL quizzes worth 10% all together. In both years, the students did very well on this quiz with class averages of 4.6/5. The questions were based on the tutorial content as well as the in-library session, so the answers could not be guessed by any student who had not completed the tutorials or had not attended the sessions.

Discussion
At the end of the first offering of the CARL course in the 2010 fall term, medical students and course designers recommended that the online tutorials not exceed one hour of independent study time, which was the case with the Evidence: Search and Rescue online tutorial. This in part was caused by the extensive list of tasks for the students created by enthusiastic librarians and navigation issues within the tutorial. In the 2011 fall term, the tutorials were shortened by decreasing the number of tasks by half, while ensuring that the remaining tasks addressed the learning outcomes. Also, the tutorial was improved by having the web-based sources open in a new window, which allowed for the easy return to the tutorial.
First-year medical students begin their program with varied academic and research backgrounds. The online tutorials and corresponding hands-on library sessions introduced new resources and skills to many students and provided a review for others with the opportunity to enhance medical expertise. The skills and medical information learned matched the requirements of the Scholar competencies and selected Medical Expert presentations. The innovative, interactive online tutorials offered an opportunity for self-directed learning and provided students with knowledge of and access to authoritative medical resources such as e-books, point-of-care tools, and medical databases. A structured and stepwise approach to the process of information access and database searching gave the students a framework to apply these skills to any other information need they will have during medical school and in their future clinical practice. Student performance and student engagement have increased and student evaluations have been positive.
The success of the blended learning innovation was a result of the extensive collaboration between the key stakeholders: librarians, medical faculty, School of Medicine educational specialists, and medical students. This group will make iterative changes to the online tutorials and facilitated in-class sessions annually to keep current with changes in resources, medical knowledge, and technologies while taking into account the medical students' course evaluations. The use of this teaching methodology is expected to be adopted in other curriculum-integrated information literacy sessions at the Queen's University Faculty of Health Sciences.