Fig. 2: Years of experience as a librarian and involvement in EBM

Librarian Role in EBM Instruction
Librarians involved with teaching EBM may take on a variety of roles in addition to that of a guest lecturer. These roles include course instructor (n=2/7, 29%), course designer or course committee member (n=4/7, 57%) embedded librarian within their programs (n=3/7, 43%) and participant in course assessment (n=4/7, 57%) (Table 1). 

Table 1: Roles of medical librarians in teaching EBM

For librarians involved with assessment, multiple choice questions were the most common assessment method used (n=3/7, 43%), while 2 respondents indicated that they use course assignments (29%).
A majority of the librarians stated that they co-create the EBM content (n=5/7, 71%) with either another health sciences librarian (n=1/7,14%), a clinical or academic faculty member (n=3/7, 43%), or both (n=1/7, 14%). When it comes to delivering the instruction, 3 respondents (43%) stated that they co-deliver the session with either a hospital librarian (n=1/7, 14%) or with clinical or academic faculty (n=2/7, 29%). Four respondents (57%) delivered EBM sessions on their own.

Setting, Educational Strategies, and Content
EBM instruction involving librarians takes place in a variety of settings including university classrooms (n=6/7, 86%), the library (n=4/7, 57%), hospital lecture halls (n=1/7, 14%), and hospital library computer labs (n=1/7, 14%). Total face-to-face contact hours between librarian and students, from all formal EBM instruction in one academic year, ranged from 2 hours (n=2/7, 29%), 3 hours (n=4/7, 57%) to 4 hours (n=1/7, 14%).
While in-class lecture is still the most common educational strategy used, with all 7 respondents (100%) saying they use this method, nearly half (n=3/7, 42%) of respondents indicated that online modules are also part of their overall instruction (Table 2). When asked about the specific teaching strategies used, didactic lecture was the most commonly reported method (n=5/7, 71%), however it was often used with other strategies such as active learning in a classroom (n=4/7, 57%) or active-learning clinically integrated (n=1/7, 14%).

Table 2: Educational and teaching strategies used in EBM instruction

71% (n=5/7) of survey respondents indicated that their EBM learning objectives were based on a competency framework; furthermore, all of those respondents (n=5/7) mentioned the CanMEDS framework.
In terms of the resources covered during EBM instruction, PubMed and the Cochrane databases were mentioned the most (n=5/7, 71%). All respondents at this point of the survey (n=7/7, 100%) incorporated either MEDLINE or PubMed in their EBM instruction. Clinical tools were also heavily featured and were reported by 5 respondents; DynaMed alone (n=2/7, 29%), UpToDate alone (n=1/7, 14%) or both DynaMed and UpToDate (n=2/7, 29%) (Table 3).

When describing the information literacy skills being taught in the sessions, all of the respondents (n=7/7, 100%) covered skills relating to the first two steps of EBM (ask and acquire) with 2 respondents (29%) also stating that they incorporate critical appraisal (step three of EBM: appraise). In addition to the formal instruction, most librarians (n=6/7, 86%) mentioned that online research guides using a platform called LibGuides were also used to support students learning EBM.
The open responses to the question asking whether significant changes had occurred in the last three years provided some insight on how often changes occur. Overall, 2 respondents (29%) stated that no significant changes had occurred, whereas the remaining 5 respondents (71%) said that significant changes had occurred in the past three years. These changes included: new course content, changes in the role of the librarian in terms of their involvement in course assessment, annual evaluation and changes made to the course and content, or a shift towards more online modules.

Table 3: Information resources being incorporated in EBM instruction

Discussion

Librarian involvement in formal EBM teaching and instruction
EBM best fits into the CanMEDS competencies within the Scholar Role because of the key concept of “Evidence-informed decision making”, as well as into the MCC Scholar objective 2: “[a]pply principles of research and information management to learning and practice” [5,6]. Both the CanMEDs concepts and MCC objectives have specific sections related to information literacy skills. Medical librarians’ competencies in information literacy are therefore well aligned to teaching some of the skills needed to practice EBM. Furthermore, as we have already established, many medical librarians are involved with EBM curricula [11,12,15]. All respondents of this survey confirm that EBM is being taught at their institutions; however only 7 out of the 11 respondents (representing 7 of 10 unique institutions) indicated that they participate in formal EBM instruction.
We defined formal instruction based on GREET [13] and also defined what we meant by a formal environment. This distinction between formal education support and other EBM education support is important because this survey did not capture the other ways medical librarians support their EBM education curriculum through consultations, optional workshops, or online research guides.
Our data did not appear to associate familiarity with the curriculum or years of experience as a medical librarian (which may not correlate with years of experience as the UME librarian) with being involved in EBM instruction; this suggests that there may be program specific or external factors impacting librarian involvement in EBM instruction. This survey did not explore these other factors, and this is an opportunity for further exploration. The comments to the open-ended questions regarding changes that have occurred within UME programs over the last three years showed that in majority of cases, changes had recently occurred or were occurring on an on-going basis. Therefore, involvement or lack of involvement of UME librarians in EBM instruction in one year may not be indicative of a trend. The changes mentioned do suggest the fluid and iterative nature of librarian involvement in EBM within the UME program. This survey did not investigate how these changes impact librarian workload, as curriculum pieces shift and teaching modalities change, and this is another area of future exploration.

Integrated approach
Gagliardi et al. discussed an EBM course that was initially offered as a noncredit elective, but eventually became a credit-bearing elective available to 3rd and 4th year medical students [8]. Liabsuetrakul et al. [16] discussed a longitudinal EBM curriculum embedded into the program. All 11 respondents to our survey stated that EBM training is formally offered at their institutions; however, only 3 respondents indicated that EBM is currently being taught as a stand-alone course. This suggests that EBM instruction is being integrated into other components of UME programs.
Survey respondents selected pre-clerkship only (n=4), clerkship only (n=1), both pre-clerkship and clerkship (n=2), or woven throughout the program (n=4) for describing the timing of formal EBM instruction at their institution. If we merge the values of those respondents who selected the option of weaving the formal instruction throughout the program, with those who selected distinct times of the program, then it appears that 10 (91%) respondents from 9 unique institutions have formal EBM instruction in pre-clerkship years, and 7 (64%) respondents from 7 unique institutions incorporate formal EBM instruction during clerkship years. Although this study does not explore the reasons for this discrepancy, the differences between pre-clerkship and clerkship curriculums structures, and the distributed nature of some clerkship programs could be contributing factors. Furthermore, the type of information taught during these sessions could be seen as foundational information that is best suited for the pre-clerkship curriculum. Our results do however, match those of Nevius et al. [1] who showed that curriculum-integrated library instruction occurs more frequently in year 1 and least frequently in year 4. Their geographic analysis (10 Canadian respondents, out of 73 total respondents) showed that Canadian libraries were more likely to report integration in year 1 (p=0.037). However, despite these results appearing to be significant, Nevius et al. [1] did caution that it is possible that the results may have been “essentially random.”

Librarian collaboration and educational approaches
Dorsch & Perry [11] discussed the common interests and collaboration that occurs between health sciences librarians and medical educators. Our data illustrates that this collaboration occurs at Canadian institutions as well. Canadian UME librarians are embedded into UME programs in order to support EBM instruction in Canada; a majority of librarians teach EBM as guest lecturers; many also sit on EBM-focused course committees and contribute to the design of the course. Almost one-half of respondents co-create and/or co-teach EBM content along with medical school faculty. This indicates that there is a significant amount of collaboration between librarians and UME programs. Our results match the trend reported in a review by Maggio & Kung[9]; of the 12 studies reported in their review, 33% (n=4) of the interventions were co-taught by librarians and clinicians, 42% (n=5) were taught by a librarian alone, 17% (n=2) were taught by only medical school faculty, and they were unable to determine who taught the sessions in one study (8%). Librarian involvement in the development and design of EBM course content is not new, and does not consist of only the individual sessions taught by librarians but can also be part of the whole EBM course. Gagliardi, Stinnett & Schardt [8] reported on an EBM course that was co-developed by a librarian and a clinician; initially it was a non-credit 6 session course, but eventually became a credit-bearing elective course for 3rd and 4th year medical students.
Active engagement in the learning process was shown to be a significant variable leading to improvement in knowledge [17]. Maggio & Kung [9] reported in their review that most studies did not provide enough detail to report on the teaching modalities or learning activities. Our survey shows that Canadian UME librarians involved in EBM instruction are utilizing active learning approaches and incorporating interactivity either in conjunction with didactic modes of teaching or using flipped classroom approaches.
Our survey highlighted the variety of resources that are being incorporated into EBM instruction including clinical tools, clinical guidelines, ACP Journal Club, and so on. However, all respondents to our survey also reported incorporating either PubMed or MEDLINE, which matches the findings reported in a 2014 review [9], where 11 (of the 12 interventions) incorporated either PubMed or MEDLINE. This result indicates that teaching how to search the biomedical database MEDLINE still represents a critical element of librarian led EBM instruction, and is being taught as part of EBM content across medical schools in Canada. Highly processed information such as the evidence-based topic summaries found in clinical tools, such as UpToDate, DynaMed or BMJ Best Practice were also frequently mentioned by our respondents. As both subject databases, such as MEDLINE, and evidence-based resources such as UpToDate, are unique and important for educational and clinical environments, this result may not be surprising. This may indicate that there is a breadth to librarian led EBM instruction. For example, searching and selecting a study from a database such as MEDLINE requires different skills than those required to search and consume an evidence-based summary such as a topic summary in UpToDate. However, similar to our survey, many studies reported in the review [9] also incorporated other information resources including pre-appraised resources such as DynaMed, UpToDate, AccessMedicine, ACP Journal Club, Cochrane reviews, and so on.
While clinical tools and Cochrane databases seem to be the ones more commonly taught as part of EBM, there is no consistency across all institutions. This should not be equated to a lack of consistency in terms of the breadth of information resources that each medical student in Canada is exposed to during their medical education. The variation may be due to many factors including: the specific questions asked in our survey, how each respondent interpreted what does or does not count as “EBM-related,” the amount of time allocated to the librarian for teaching EBM (more allocated teaching time may lead to a greater number of information resources being incorporated), or the way in which the medical program has been designed (it may be that other resources are incorporated in other parts of the program).
In addition to participating in EBM instruction, our survey also showed that Canadian librarians create online research guides (n=6/7, 86%) and online modules (n=3/7, 43%) for EBM instruction. These types of asynchronous electronic resources can be used by students at any time of day and throughout their time as a medical student. Our results are similar to those of Nevius et al. [1] who reported that 85% of the responding medical libraries created subject guides.
While only a minority of librarians created EBM online modules for their students, the open-ended comments received as part of the responses in our survey expressed the need for an increased use of online modules, particularly in programs with regional medical campuses. Nevius et al. [1] reported that 78% of responding medical libraries created recorded tutorials. Their number is higher than that found by our survey, although this is not surprising as our survey focused specifically on EBM, whereas they were reporting on all librarian-led instruction, which incorporates EBM as well as other topics such as databases, citation managers, or apps, to name a few. Online modules have benefits; they do not have to occur in a class and students can complete the online modules at their own pace and re-watch as needed. However, online modules are time consuming to create, need technological or financial support, and also require the development of proficient e-learning best practices. Schilling [18] examined the impact of traditional versus e-learning on the information retrieval skills of first-year medical students. They gathered data using pre and post skills and attitudes surveys and included an evaluation of the students’ MEDLINE search strategies. Their study showed no significant differences in the MEDLINE searching scores between the two groups (traditional classroom and e-learning).

Limitations
There were several limitations to our study. In the spirit of transparency, we have attempted to share all known limitations, and recognize that many improvements can be made in future research.
Our survey had a lower than desired response rate of 59%. With seventeen accredited medical schools in Canada, we received responses from librarians representing only ten different medical schools. Nevius et al. [1] surveyed both US and Canadian medical libraries, and received a total response rate of 47%, which included responses from ten Canadian libraries (matching the number of responses we received to our survey). We did not translate our survey into French, and this may have affected the response rate.
A further limitation of the cross-sectional study design is that it only shows a snapshot of that particular point in time; in this case capturing data from 2017-2018, the academic year preceding the distribution of the survey.
The population targeted for this survey was only Canadian academic medical librarians participating in EBM instruction within the UME curriculum. Other librarians, such as hospital librarians, may be involved with EBM instruction in UME programs, especially during clerkship, through activities such as orientations, clinical rounds, or as community stakeholders on EBM projects. This survey does not capture their work, and only represents data from when an academic medical librarian was responsible for organizing or leading the instruction.
Respondents were asked how long they had been a medical librarian, but were not specifically asked how long they had been involved with the UME program. It is unknown whether this will impact the likelihood of being involved in teaching EBM within the UME curriculum. Furthermore, some questions asked about the respondents’ practices regarding teaching, creation and delivery of content, selection of resources included, and so on, whereas other questions specifically asked about the respondent’s institution. Therefore, it should be noted that some responses reflect the practices of the respondents and are not reflective of institutional practices.
The split path in our survey prevented those respondents who were not involved in EBM instruction from continuing on to respond to the open-ended questions. This prevented us from receiving some potential responses about significant changes that had occurred in the three previous years. This information could have provided some insight into whether changes in involvement had occurred in recent years, whether that year was an anomaly due to extenuating circumstances, or whether the lack of involvement was consistent over the previous few years. As the survey was anonymous, it was not possible to follow up with respondents who exited the survey at the split path to obtain responses to the open-ended questions. Future research using a similar design should take this into consideration when designing surveys.
The three investigators involved in this study met the eligibility criteria to participate in the survey. Since the intention of this survey was to highlight the participation of academic medical librarians in EBM instruction within the UME curricula in as many of the seventeen Canadian medical schools as possible, the investigators also responded to the survey as they were the only representatives for their individual institution or site. Since this survey’s intent was to report on the specific aspects of librarian activities which were considered to be objective rather than subjective, the investigators determined that the benefits of including data from their own practices and their own institutions would outweigh the risk of bias.

Future directions
Clerkship was identified from the responses to our survey questions as an area of opportunity and further collaboration. This sentiment was expressed through comments and responses to the open-ended questions, where librarians stated that they would like to pursue more co-teaching opportunities, as well as collaboration with clinical faculty during clerkship. Identifying potential barriers or successes in achieving further collaboration during clerkship is an area of future research that may be of interest to medical librarians.
This study did not investigate the similarities or differences involved in librarians teaching EBM in person compared to virtual environments. However, EBM instruction using online modules was expressed as an emerging need by several survey respondents. Further research on the incorporation of virtual learning as part of EBM instruction is needed. As well, future research should consider a mixed-methods approach using phone interviews in addition to a survey in order to capture more in-depth information on librarian involvement with EBM instruction within the UME curriculum.
Assessment is an important component of teaching and learning, and plays an important role in UME curricula. This study did not focus on the nature of the assessment, and only asked one split question about librarian involvement in assessment. Investigation into librarians’ involvement with assessment in EBM curricula in UME is needed.
As familiarity with the curriculum or years of experience did not appear to correlate with involvement in EBM instruction, further research is required to explore what other factors created opportunities and barriers for librarian involvement in formal EBM instruction within UME programs.

Conclusions
The results of this survey provide evidence of the activities and practices of some UME librarians in Canadian medical schools, and could be useful for other UME librarians in similar contexts who are considering changes to their teaching practices or are trying to advocate for different roles than those they are currently involved in. The survey specifically focused on librarian involvement in formal EBM instruction within the UME curricula. The results of the survey demonstrated that these librarians are embedded within the EBM curricula, with instruction ranging beyond the first two steps of the EBM curriculum, and teaching using a variety of educational strategies, including active learning as well as the use of online modules. Many of the librarians involved within the UME curriculum collaborate with other librarians as well as with medical faculty. The results of this survey highlight the dynamic and fluid nature of librarian involvement in EBM within UME programs in Canada.