Research Article
The Evidence Synthesis Institute: Building a High Impact Training Program and Supportive Community for an Emergent Area of Librarianship
Megan Kocher
Science & Evidence Synthesis Librarian
University Libraries
University of Minnesota
Minneapolis, Minnesota, United States of America
Email: mkocher@umn.edu
Amy Riegelman
Social Sciences & Evidence Synthesis Librarian
University Libraries
University of Minnesota
Minneapolis, Minnesota, United States of America
Email: aspringe@umn.edu
Matthew Kibbee
Evidence Synthesis Coordinator
Albert R. Mann Library
Cornell University
Ithaca, New York, United States of America
Email: mk282@cornell.edu
Molly Bostrom
Biology Librarian, Sciences, Agriculture, and Engineering
University Libraries
University of Minnesota
Minneapolis, Minnesota, United States of America
Email: mbostrom@umn.edu
Scott Marsalis
Director, Sciences, Agriculture & Engineering
University Libraries
University of Minnesota
Minneapolis, Minnesota, United States of America
Email: marsa001@umn.edu
Sarah Young
Social Sciences Librarian, Director of the Evidence Synthesis Program
University Libraries
Carnegie Mellon University
Pittsburgh, Pennsylvania, United States of America
Email: sarahy@andrew.cmu.edu
Zahra Premji
Health Research Librarian
Mearns Centre for Learning - McPherson Library
University of Victoria
Victoria, British Columbia, Canada
Email: zahrapremji@uvic.ca
Lindsay Bahureksa
Research & Evidence Synthesis Librarian
Albert R. Mann Library
Cornell University
Ithaca, New York, United States of America
Email: lab384@cornell.edu
Natasha Bishop
Evidence Synthesis Information Scientist
University Libraries
Virginia Tech
Blacksburg, Virginia, United States of America
Email: nebishop@vt.edu
Gracian Chimwaza
Director
Information Training and Outreach Centre for Africa (ITOCA)
Centurion, Gauteng, South Africa
Email: gracian@itoca.org
Cozette Comer
Assistant Director, Evidence Synthesis Services
University Libraries
Virginia Tech
Blacksburg, Virginia, United States of America
Email: cozette@vt.edu
Elizabeth DiGiovine
Research & Evidence Synthesis Librarian
Albert R. Mann Library
Cornell University
Ithaca, New York, United States of America
Email: emd263@cornell.edu
Received: 3 Mar. 2025 Accepted: 25 Aug. 2025
2026 Kocher,
Riegelman, Kibbee, Bostrom, Marsalis, Young, Premji, Bahureksa, Bishop,
Chimwaza, Comer, and DiGiovine. This is an Open Access article
distributed under the terms of the Creative Commons‐Attribution‐Noncommercial‐Share
Alike License 4.0 International (http://creativecommons.org/licenses/by-nc-sa/4.0/),
which permits unrestricted use, distribution, and reproduction in any medium,
provided the original work is properly attributed, not used for commercial
purposes, and, if transformed, the resulting work is redistributed under the
same or similar license to this one.
DOI: 10.18438/eblip30736
Abstract
Objective – Seeing an absence of training opportunities for librarians supporting evidence synthesis outside of the health sciences, a collaborative team of librarians sought grant funding, developed a curriculum, and facilitated community building to provide this training.
Methods – The present case study describes the identification of a training gap, multi-faceted efforts made to fill that gap, and evaluation via pretests and post-tests to continuously improve the program.
Results – The Evidence Synthesis Institute (ESI) was launched in 2020. An application process is used to select the 50-person ESI cohort, with an average acceptance rate of 37%. To date, over 500 individuals have attended the Evidence Synthesis Institutes. Pre- and post-institute surveys show consistent gains in confidence to perform a variety of evidence synthesis skills. Additionally, we developed a self-paced online course which has over 750 enrolled learners with 118 having completed at least 90% of the course.
Conclusion – This case study provides a model for designing, implementing, and assessing a training program and community of practice for an emergent skillset in academic librarianship. As shown by the program’s acceptance rate, demand remains high for participation in evidence synthesis training. Future programming could build on past success, such as continuing to offer synchronous virtual Institutes, training additional instructors, and expanding the asynchronous online course to include more content and summative assessments.
Introduction and Background
Introduction to the Evidence Synthesis Institute
Evidence synthesis (ES) yields the strongest evidence of any research design and serves as the gold standard for informing practice, future research priorities, policy-making, and high-level decision-making. Evidence syntheses rely on a comprehensive search for evidence, and their quality can be greatly improved when a research librarian trained in evidence synthesis methodologies plays an active role in the research process (Aamodt et al., 2019; Koffel, 2015; Meert et al., 2016; Pawliuk et al., 2024; Rethlefsen et al., 2015). Therefore, it is imperative that reputable, up-to-date, and accessible training opportunities and continuing education exist for librarians.
The Evidence Synthesis Institute (ESI) was created to fill gaps in precursory training programs, such as a lack of training for review methodologies beyond systematic reviews, minimal focus on disciplines outside of the health sciences, and limited programming that is both accessible and affordable. ESI currently exists as a free four-day synchronous training program that takes place twice per year. Since summer 2024, parts of it are also available as an asynchronous online course. ESI also supports a community of practice and professional development opportunities for participants. This article details the development and implementation of ESI as a case study in meeting an emergent need in librarianship for skill development and community support through identifying a need, piloting trainings in multiple formats, iteratively improving upon materials, empowering participants to use course materials to train colleagues, and intentionally creating a community of practice in partnership with an existing professional organization. This model can be built on and adapted to fill other training gaps in academic librarianship as user needs are constantly evolving.
Evidence Synthesis & Librarian Support Outside the Health Sciences
Evidence synthesis methodologies have been used in disciplines beyond the health sciences (Chalmers et al., 2002), and their use continues to grow. In the past decade, the number of non-medical publications using evidence synthesis methodologies increased from 2,131 (10% of total evidence synthesis publications) in 2014 to 13,894 (16.2%) in 2023 (Bahureksa, 2024). In a recent study (Slebodnik et al., 2022), the top five primary subject areas that published the greatest number of non-health ES reviews were social science, environmental science, business, computer science, and engineering.
Many cross-sectional or benchmarking studies show that librarians in disciplines outside the health sciences have been receiving a growing number of ES-related requests. Lê et al. (2024a) surveyed librarians who serve non-health disciplines, and their results showed researchers in the social sciences (39.3%) and sciences (18%) most frequently requested ES support. Furthermore, 80 of 81 respondents who received requests reported an increase in overall requests in the last five years. Other studies have focused on the roles of librarians in ES in specific disciplines, including conservation biology (Boice, 2019); social work (Marsalis, 2020); sustainable development (Ghezzi‐Kopel et al., 2022); mathematics education (Kogut et al., 2022); business (Premji et al., 2022); and the sciences, humanities, and social sciences (Lê et al., 2024b).
Guidance from evidence synthesis organizations have highlighted the fundamental role that information specialists and librarians fill as members of an evidence synthesis review team (Aloe et al., 2024; Aromataris et al., 2024; Higgins et al., 2023). There is also a growing body of literature that demonstrates the positive impact of librarian involvement on the completeness of reporting or quality of reviews (Aamodt et al., 2019; Koffel, 2015; Meert et al., 2016; Pawliuk et al., 2024; Rethlefsen et al., 2015).
Existing opportunities for librarians seeking to learn how to conduct and support evidence synthesis are limited in that there is no in-depth evidence synthesis training offered as part of MLIS programs (Lafferty et al., 2024). The training opportunities that do exist are developed for health science librarians and researchers. Medical librarians have access to a variety of high-quality training programs and resources to support evidence synthesis work, including the University of Michigan’s Systematic Review Workshop and the Medical Library Association’s Systematic Review Services Specialization (Medical Library Association, n.d.; University of Michigan Library, n.d.). In addition to training programs, medical librarians have access to a well-established community of experienced librarians via listservs, fora, and active social media accounts devoted to systematic reviews. Although some skills learned in these programs and networks might be relevant to librarians working outside the health sciences, these programs tend not to address the specific databases, methodological challenges, or intricacies of information retrieval specific to non-medical disciplines. Exacerbating the gap, what few programs exist often have significant financial costs due to registration fees or in-person travel requirements. Prior to 2020, no programs in the U.S. offered free, virtual, and synchronous evidence synthesis training for librarians with a multidisciplinary focus, and a recent survey showed that librarians consider insufficient training to be a barrier to providing assistance on evidence synthesis projects (Boden et al., 2025).
Beyond training, cross-disciplinary communities of practice or networks for librarians who support evidence synthesis were scarce. This is a fundamental piece of the development of evidence synthesis librarianship as training alone cannot meet the growing demand for high-quality searches in ES reviews. There are several communities for health and medical librarians focused on evidence synthesis, including the Medical Library Association’s expertsearching listserv and the Canadian Health Libraries Association’s Knowledge Synthesis Interest Group (Expertsearching, n.d.; Knowledge Synthesis Interest Group, n.d.).
Small constituencies of information specialists in disciplines outside of health and medicine have formed communities, such as the Information Retrieval Management Group (IRMG) in the Campbell Collaboration, which focuses on systematic reviews and mapping reviews in social science. Although there is a meaningful difference in how evidence synthesis is approached across disciplines, the core principles of these methods (e.g., comprehensiveness, transparency) are consistent. The lack of a community that allows members to focus on the needs of librarians in evidence synthesis across disciplines presents an opportunity for disjunct development in terms of practice and even standards for searches in evidence synthesis.
Building the Evidence Synthesis Institute
Due to the lack of options for training, institutions and professional organizations for librarians in disciplines outside the health sciences initially looked to peers to provide training for members interested in evidence synthesis skills. In 2018, members of the University of Minnesota and Cornell University evidence synthesis service teams were invited to the National Agricultural Library in Beltsville, Maryland to provide a 2-day systematic review training for their librarians. They were also invited to present a pre-conference workshop on systematic reviews at the United States Agricultural Information Network conference that year. It became clear that these types of requests would become more frequent and place an undue burden on the few people able to provide training in addition to doing their full-time jobs.
As a result, this small group of colleagues looked to combine knowledge, distribute work, and build a professional community, which led to the initial plans for the Evidence Synthesis Institute (ESI). We envisioned a free, librarian-centered evidence synthesis training program that would provide information professionals with the skills, confidence, and sense of community they would need to support evidence synthesis research at their own institutions. The core components of this vision were not specific to evidence synthesis and would translate well into a flexible, person-centered, cost-effective framework for other specialized librarian training.
Grounding Values
ESI came about at a unique time, launching in 2020 amidst the global COVID-19 pandemic. Because of this, it was especially important to us that the grounding values of the program be centered on a holistic, person-centered approach. This approach recognizes the vulnerability and needs of each person involved—both instructors and participants—and guides our practices in terms of openness, flexibility, and compassion. In practice, this led to the development of a code of conduct to set expectations for behavior and treating each other well, allowing participants to switch cohorts when unexpected life events made completion difficult, cross training of instructors to cover for each other when needed, not recording sessions so that both instructors and participants felt comfortable being candid and expressing insecurities, and encouraging a culture of openness and curiosity with multiple modes of submitting questions in real time and anonymously. This principle underlies not just ESI training but also the professional community we have established to be a supportive environment.
Another grounding value of the Institute was to have all our materials open and available to all. This was to maximize the impact and reach of our materials and to empower our learners to become instructors themselves. With these resources, discussed in more detail below, ESI participants received a supportive set of documents to refer to after the training and a foundation of instructional materials that they can use and adapt in their own teaching.
Cost to participants was also a factor in ensuring the Institute was accessible regardless of an individual’s access to funding. Initially, the plan was to provide the in-person training free of charge while covering the travel and accommodation costs of the participants and instructors. With the pivot to a virtual synchronous Institute due to the COVID-19 pandemic, we were able to provide training at no cost to participants and direct the funding originally earmarked for participant travel costs to provide other ESI events at low or no cost to participants.
Pilot Institutes
In April 2019, a small, in-person, 2-day training occurred on the University of Minnesota campus. It was marketed as a pre-conference for the Minnesota Chapter of the Association of College and Research Libraries annual conference. This endeavor was organized by librarians from the University of Minnesota and Cornell University. We built a scaffolded curriculum and set the maximum number of attendees at 15. The training and some meals were free for attendees, but attendees were responsible for travel and lodging.
Following the successful completion of the in-person Institute at the University of Minnesota, and in consideration of the COVID-19 pandemic, we next sought to experiment with offering the training on a virtual platform. Cornell University Library coordinated this training over three days in August 2020. This required adapting activities and timelines to suit the virtual format. It also allowed for more participants (50 instead of 20) and more instructors. The agenda and recorded sessions from this pilot are available online at https://guides.library.cornell.edu/EvidenceSynthesisTraining.
IMLS Grant
Building on what we learned from the pilot trainings, Megan Kocher and Amy Riegelman, along with collaborators from the University of Minnesota Libraries, Cornell University Library, and Carnegie Mellon University Libraries, applied for the Laura Bush 21st Century grant from the Institute for Museum and Library Services (IMLS). The grant application outlined the dearth of ES training opportunities for librarians outside of the health sciences and detailed how we intended to fill that gap by designing training for 120 librarians via in-person opportunities over three years. The grant objectives were:
● leverage the expertise of project partners to develop the pilot curriculum into a full 2.5-day curriculum that thoroughly covers the topics, engages participants, and incorporates a variety of teaching and learning modalities
● develop an online toolkit of resources for librarians supporting evidence synthesis outside the health sciences that can be shared and used by the library community at large
● train a cohort of 120 librarians in evidence synthesis and provide them with the tools to share this knowledge with others
● use feedback from the trainings to rework the curriculum and establish the structure to offer and potentially expand the training program after the grant period
(Kocher et al., 2020)
We obtained letters of support for the grant application from each of the partner institutions, the director of the National Agricultural Library, and from prominent health science evidence synthesis experts at the University of Florida Health Science Center Libraries, the University of Michigan Taubman Health Sciences Library, and the Texas A&M University Medical Sciences Library.
In July 2020, we were notified that IMLS would fund this work beginning in October 2020, but the pandemic drastically changed how we could implement the objectives of the grant. We worked with IMLS to allocate the existing funding in different ways and moved to tailor our instruction to a virtual environment. Funding was redirected to support other aspects of the Institute, including instructional design, honoraria for a wider team of instructors to deliver the content, production of a series of videos to supplement virtual instruction, and high-quality microphones to support an accessible experience of extended online listening and participation. We were eventually approved for a grant extension to spend the remaining funds on two additional Institutes, a hybrid conference, development of a free self-paced online course, and a facilitated strategic planning retreat.
Recruitment and Application
The intended audience for ESI is librarians and information specialists supporting evidence syntheses in disciplines outside of the health sciences. We cap attendees at 50 participants per cohort to keep the instructor:student ratio low and encourage active participation and community building. To facilitate the application process and recruit participants, we use a digital application form with a corresponding rubric and advertise on various association listservs and in College & Research Libraries. As indicated in the IMLS grant proposal diversity plan, we are "committed to inviting and welcoming librarians representing diverse organizations, backgrounds, and experiences to participate," and we post to listservs such as HBCU Library Alliance, American Indian Library Association, and Ethnic & Multicultural Information Exchange Round Table (Kocher et al., 2020).
We also host an e-form on our ESI webpage where interested individuals can register to receive email updates about future ESIs (Evidence Synthesis Institute, n.d.). Application windows are typically open for three weeks. Applications are scored and prioritized based on urgency of need, existing knowledge gap, disciplines supported by the applicant (since we are dedicated to supporting reviews outside the health sciences), and potential impact of the training. All application screeners participate in normalizing exercises to reduce bias and improve inter-rater reliability of screening decisions. After screening, the 50 top scoring applicants are notified that they have been accepted to ESI.
Assessment
We ask participants to complete anonymized surveys before each Institute, immediately after the Institute, and six months after completing the Institute. The surveys consist of seven questions asking participants to self-assess their ability to perform certain aspects of evidence synthesis on a 5-point scale (see Appendix A for the survey instrument). These questions address key learning objectives for ESI related to different steps of ES and roles of librarians in supporting ES.
In addition to questions about ES steps, there is also a question asking participants about their perception of their library’s readiness to establish an ES service. While this is not an issue that ESI can address directly, we were interested in understanding changes in confidence in regards to service development, since our hope is to empower librarians and library leaders to establish formal services. In the post-Institute survey, we also ask open-ended questions about useful and least useful aspects of ESI topic coverage and timing, which contribute to our continual improvement of content and delivery.
The 6-month survey aims to understand longer-term applications of learning and knowledge retention. This is assessed with questions about how participants have used the knowledge gained at ESI, what ESI materials they have used post-Institute, how they have continued to learn about evidence synthesis, and barriers to participating in ES projects.
Of the 350 participants surveyed to date, 249 have answered both the pre- and post-Institute surveys (71% response rate), and 161 have answered the 6-month survey (46% response rate). The pre-post surveys consistently indicate statistically significant gains in confidence to perform all ES tasks (Table 1). Across the different iterations of the Institute, we used these pre-post survey results and the free text comments provided by participants to improve the ESI curriculum. For example, in early iterations of ESI, we implemented a series of breakout rooms to work on search strategy development skills in subgroups. We noticed a decrease in improved confidence in searching-related survey items, and participants noted that the time allotted to breakout room activities was insufficient to complete the tasks. At the same time, participants reported relatively high confidence in using citation management tools in both pre- and post-Institute surveys. Based on this, we flipped the citation management content and offered a brief video that could be viewed outside of ESI, which provided more time in the curriculum for search-related breakout room activities. With more time for search strategy content, subsequent cohorts reported higher gains in confidence related to searching.
Table 1
Paired t-tests Comparing Pretest and Posttest Scores for Perceived Self-efficacy Related to Evidence Synthesis
|
Survey Question |
Pretest Mean (SD) |
Posttest Mean (SD) |
t value |
Effect Size (Cohen's d) |
|
I am able to guide researchers in developing well-defined research questions for evidence synthesis. |
2.45 (0.97)
|
3.96 (0.70)
|
24.25***
|
1.54 |
|
I am able to design reproducible searches to comprehensively find existing research to address a well-defined research question. |
2.90 (1.03)
|
3.91 (0.75)
|
15.91***
|
1.01 |
|
I can identify an appropriate set of bibliographic databases to search for an evidence synthesis project in my subject area. |
3.75 (0.95)
|
4.45 (0.67)
|
11.75*** |
0.75 |
|
I can effectively use database-specific syntax to translate a search strategy across multiple sources. |
3.16 (1.04)
|
3.88 (0.81)
|
11.38***
|
0.72 |
|
I can find and identify evidence synthesis guidelines and standards relevant to different methods and disciplines. |
2.37 (0.95)
|
4.16 (0.76)
|
26.92*** |
1.71 |
|
I can navigate the various options for citation management and screening to manage the evidence synthesis process. |
2.82 (0.95)
|
3.96 (0.86)
|
17.75***
|
1.12 |
|
My library is prepared to offer evidence synthesis support as a service to researchers at our institution. |
2.81 (1.24)
|
3.17 (1.07)
|
4.97***
|
0.32 |
1=strongly disagree to 5=strongly agree; df = 248; asterisks denote level of significance (* p < .01, ** p < .001; *** p < .0001)
The 6-month surveys indicate mixed retention of confidence in ES skills over time (Table 2). There have been a large range of responses in regards to application of skills gained at ESI and barriers to engaging in ES activities. There are some participants who had limited opportunities to apply knowledge from ESI, thus likely losing confidence in skills, and others who had engaged deeply in ES activities, including co-authoring evidence syntheses and mentoring others in evidence synthesis. More research is needed to determine the types of activities that would help with knowledge retention and how to address organizational and other barriers to applying ES skills in the workplace.
Table 2
Changes in Perceived Self-efficacy 6 Months Post-Institute
|
Survey Question |
Posttest Mean (SD) |
6-months Mean (SD) |
t value |
Effect Size (Cohen's d) |
|
I am able to guide researchers in developing well-defined research questions for evidence synthesis. |
4.04 (0.70) |
3.72 (0.75) |
-4.50*** |
-0.459 |
|
I am able to design reproducible searches to comprehensively find existing research to address a well-defined research question. |
3.98 (0.71) |
3.73 (0.81) |
-3.03* |
-0.309 |
|
I can identify an appropriate set of bibliographic databases to search for an evidence synthesis project in my subject area. |
4.52 (0.62) |
4.43 (0.71) |
-1.22 |
-0.124 |
|
I can effectively use database-specific syntax to translate a search strategy across multiple sources. |
3.88 (0.82) |
3.84 (0.87) |
-0.41 |
-0.042 |
|
I can find and identify evidence synthesis guidelines and standards relevant to different methods and disciplines. |
4.27 (0.76) |
3.96 (0.74) |
-3.97** |
-0.405 |
|
I can navigate the various options for citation management and screening to manage the evidence synthesis process. |
4.06 (0.79) |
3.75 (0.89) |
-3.17* |
-0.324 |
|
My library is prepared to offer evidence synthesis support at a service to researchers at our institution. |
3.15 (1.07) |
2.93 (1.10) |
-2.01 |
-0.205 |
|
Overall self-efficacy scale |
3.98 (0.51) |
3.76 (0.54) |
-4.83*** |
-0.493 |
1=strongly disagree to 5=strongly agree; df = 95; asterisks denote level of significance (* p<.01, ** p < .001; *** p < .0001)
Running ESI
Instructors for the Institute were initially recruited from the evidence synthesis teams at the three partner institutions. Team leads from each institution met to determine who would teach each section of the Institute, and instructors were responsible for creating or adapting slides and activities for their sections. Instructors met together prior to each Institute to walk through the material. In the interest of expanding our instructor base and bringing in more perspectives and expertise, we recruited additional instructors first from the group of librarians who had participated in the CERES 2030: Sustainable Solutions to End Hunger project and then from past attendees of ESI (International Institute for Sustainable Development, n.d.). Instructors used feedback from each Institute to make changes to content for the next iteration. This resulted in the emergence of a more stable curriculum after the first two years of the Institute, which now typically only needs to be updated to reflect updates to resources and guidelines.
The Institute is held on Zoom and makes use of interactive polls, breakout rooms for daily subject-area discussion and search strategy exercises, and breakout sessions on the final day. Throughout the training, we place an emphasis on an active chat for questions and general commentary from both instructors and attendees. Chat coordination falls to a virtual producer who, both in the lead-up to and during the Institute, liaises with instructors to determine the production needs of their sessions, builds and manages the polls and shifting breakout room set-ups, shares links to relevant slides and resources in the chat, and troubleshoots any technological issues participants may encounter. They also manage communication with participants, including sharing pre-work and logistical information, pre- and post-surveys, chat transcripts and homework assignments after each day of the Institute, cohort directories, and certificates of participation after the Institute; and adding participants to the ESI listserv. Prior to, during, and after the Institute, the virtual producer manages ESI’s web presence, including a repository of course materials and a website with current application information and updated instructional resources.
The 4-day agenda for the Institute was intentionally constructed to best support participant learning, both in how the content is presented and by establishing a manageable pace for both instructors and attendees. The material is scaffolded to ensure participants understand fundamental concepts and principles of evidence synthesis taught early in the Institute before building upon them to support understanding of more complex aspects of evidence synthesis covered later in the Institute. With the pivot to a synchronous virtual learning environment, we also wanted to be intentional about having regular and generous breaks throughout the Institute, generally around 1.5 hours out of each 5-hour day, to make the learning and teaching processes more productive and accessible for all participants, including instructors. We have also built in active learning opportunities with polls and questions for participants to reflect on as well as collaborative, hands-on activities around constructing search strategies. In an effort to provide participants with the chance to reflect on the day’s learning, raise questions, and discuss subject area-specific concerns with peers, we end the first three days with disciplinary breakout rooms led by instructors in those same disciplines.
To create a safe, supportive, and harassment-free space to best facilitate learning, we drafted and shared a code of conduct (CoC) with an accompanying anonymous CoC incident report form. Attendees are encouraged to report CoC violations, and ESI planning committee members are alerted if there are submissions. Form submissions are actively monitored throughout the Institute, but to date, no code of conduct violations have been submitted.
To help participants prepare for the fast-moving instruction, we assign work to complete in advance of the Institute. This includes reviewing our code of conduct and daily agenda, watching a number of videos introducing basic concepts and steps in the evidence synthesis process, and pre-reading to help lay a foundation for the rest of the Institute (umnLibraries, n.d.). In addition, we ask participants to fill out the pre-survey that, when compared with the post-survey, allows us to evaluate what is and is not working in the Institute. The feedback from these surveys has led to iterative changes around time allotted to particular activities, the Zoom and tech setup of instructors to make the training more accessible for all participants, the placement and duration of breaks, and how to make the scaffolding of information most beneficial and understandable.
One essential aspect of the Institute’s success is creating an environment that welcomes the questions of the participants and offers many ways in which they can ask those questions. In addition to the previously mentioned disciplinary breakout sessions that offer people a smaller group to discuss questions, they also have the option to, at any time, anonymously submit questions via an online form that becomes the basis for a question and answer session on the last day of the Institute. Throughout the Institute, a robust and informal Zoom chat encourages interaction from participants, who get immediate and collegial responses from both the team of instructors and their peers. These chat transcripts are then shared at the end of each day for participants to reference.
Evidence Synthesis Institute for Groups
In addition to the biannual Institutes with open calls for registration, in 2023 we produced three versions of ESI for specific institutions, albeit with mixed results. The impetus for creating these tailored versions of the Institute was inquiries from multiple institutions asking whether ESI could provide training for their entire team prior to the initiation of an ES service. To test out how this would work, we solicited applications for ESI for Groups. We received 35 applications, and after scoring the applications against a rubric, selected one large academic library, one mid-sized academic library, and one special library to participate. These were generally smaller groups of 5-20 participants who wouldn’t necessarily have self-selected to be ESI attendees if their organization had not arranged for a group Institute. Additionally, the 4-day agenda was condensed into three days. These factors led to less robust participation and less opportunity for sharing, discussion, and reflection amongst the participants. Upon reviewing the feedback from both participants and instructors for these trainings, we decided not to continue offering the group Institutes because they were a significant time investment for a smaller impact than the regular ESI trainings.
Open Content for Broader Reach
ESI intentionally developed and made materials from the Institute available in an open and reusable format. This aligned with our train-the-trainer model, which enables Institute participants to reuse and adapt materials for their own purposes, including training and workshops at their own institutions. These materials include all slides used for each module, an up-to-date resource list, videos introducing many of the steps in evidence synthesis, and a frequently updated Zotero library of curated materials organized by topic (Evidence synthesis articles, n.d.; Evidence Synthesis Institute, n.d.; umnLibraries, n.d.). These materials are collected and made available through Pressbooks, a platform for open educational resource publishing that is available through University of Minnesota.
These materials have been repurposed for an international expansion of the training. In Africa, the Information Training and Outreach Centre for Africa (ITOCA) has organized and hosted the Evidence Synthesis Training for African Librarians (Information Training and Outreach Centre, n.d.). Three institutes have been delivered to date that have trained 80 participants from the continent adapting materials developed by ESI. In Canada, University of Victoria Libraries sponsored a pilot offering of the Institute in 2022 with support from the Canadian Association of Professional Academic Librarians’ Research and Scholarship Committee. Based on the success of the pilot and continued interest from participants, and in partnership with the Canadian Association of Research Libraries, annual Canadian institutes continue to be offered (Evidence synthesis institute Canada, 2025). The Canadian institutes have trained 150 participants across three institutes, with another two planned in 2025, including the first exclusively in French language.
In addition to making Institute materials available to participants, we also saw a need for training materials that could be used by a wider audience. For this reason, we converted the ESI materials to an online, asynchronous, self-paced course. Working with an instructional designer from the Open Learning Initiative at Carnegie Mellon University (https://oli.cmu.edu/), the content from slide decks was used to build fifteen modules covering the majority of the topics addressed by ESI. The course includes formative assessments that will help us assess learning and iterate on the materials over time (Kocher et al., 2025). The course is currently freely available and allows us to greatly expand the reach of ESI to a global audience. As of the writing of this article, over 750 learners have enrolled in the course, and 118 have completed at least 90% of the course.
As with any online content, there is a need to consider the sustainability of the platforms and the maintenance and updating of the content and tools as practices in evidence synthesis evolve over time. We are developing a sustainability plan to ensure the longevity of the online materials. In addition, an accessibility assessment will be important to ensure that we are meeting accessibility standards for a diverse audience. This includes consideration of learners in under-resourced areas; for example, building materials that are friendly to low-bandwidth regions should be considered.
Building Community
Communities of Practice
To create an ongoing community of practice, several of the Evidence Synthesis Institute (ESI) founders worked with the Association of College and Research Libraries (ACRL) to create the Systematic Reviews and Related Methods Interest Group (renamed the Evidence Synthesis Methods Interest Group in 2021). ESI worked closely with the ACRL group to route participants to its listserv, resource guide, and discussion groups. This allowed for connections among librarians interested in evidence synthesis regardless of whether they had attended ESI.
A central goal of ESI has been to facilitate informal networks of librarians working in similar disciplines. Each Institute includes disciplinary cohorts of librarians grouped into business and policy, science and agriculture, and social sciences. To keep the groups small enough to allow for conversation and broad participation, more than one group for a discipline is typically offered. Each group is facilitated by one or two instructors, and conversations include follow-up on topics covered previously in the day. Participants share institutional challenges and opportunities and vendor platform challenges and ask the instructors to contextualize information to their own disciplines.
In an effort to foster both inter- and intra-cohort community, and due to an acute need to communicate about the ESI Conference, we created an email listserv of all past ESI instructors and participants in October 2023. The intention was to create a venue for people to raise their questions and have discussions on evidence synthesis. Over 50 conversations were initiated in the first 11 months of the listserv’s existence, including threads about search strategy best practices, database troubleshooting, new or updated ES resources, sharing ES-related job postings in libraries, and collaboration ideas.
ESI Symposium and ESI Conference
On May 6, 2022, we held a virtual symposium for past ESI participants to provide a community-building opportunity to share and discuss projects, initiatives, resources, challenges, and activities related to evidence synthesis work. We had 21 short sessions, all of which were recorded and the materials made available. We assembled an attendee directory of 117 participants, including their institutions and subject area specialties, which was shared with all attendees with the intention of facilitating collaboration between ESI participants with overlapping areas of interests.
Given the success of the ESI Symposium, we held the first ESI Conference on April 16-18, 2024, online and in-person at the University of Minnesota. The conference provided a space for both virtual and in-person participants to share the work they had done since participating in our workshops, learn about new or more advanced topics in evidence synthesis, and get acquainted with our open online course materials so that they can use them to host their own workshops. Attendance at this conference was limited to past ESI participants; those who attended trainings that were adapted from ESI and hosted internationally were also invited to attend the conference virtually. The conference was attended by 340 participants total—86 in-person and 254 online. To increase accessibility, the IMLS grant covered travel-related expenses for all conference presenters and for 15 in-person attendees via a lottery-based scholarship.
Discussion and Conclusion
As evidenced by our evaluations, the process described here has resulted in a highly effective and continuously in-demand training program that meets a gap previously unfilled by LIS program curricula and professional development opportunities offered by professional organizations. It is a model that could be used to fill other emergent gaps, as the skills required of academic librarians are constantly evolving to meet the needs of researchers and academic institutions. This model is rooted in a set of values that lead to accessibility and sustainability through multimodal offerings, open content, and ongoing support for trainees.
The demand for training in ES has remained high; therefore, expanding future programming would prove valuable to the greater research community. Figure 1 shows the acceptance rate for each Institute, which averages at 37% acceptance. We are consistently unable to support all interested applicants, and due to restrictions of the grant, we have especially been unable to meet the demand for training from applicants outside the United States and those from the health sciences. Although our online course now provides an option for applicants who are not accepted to ESI, demand continues to outstrip available training resources. We anticipate that more ESI instructors will need to be recruited and trained to help meet the high demand for evidence synthesis training and to update online resources to reflect best practices and current guidelines.
Figure 1
Acceptance rates for each Evidence Synthesis Institute.
By the end of the IMLS grant period, ESI had trained over 500 librarians, developed a stable and effective curriculum, and launched a basic version of an asynchronous online course. In seeking future funding, we hope to continue the successful synchronous virtual Institutes, train more instructors to implement them, and to expand the asynchronous online course with more content and summative assessments.
Moving forward, we seek to provide past attendees with continued learning opportunities to expand and reinforce the skills they have learned at ESI. Our advanced learning programming will be largely informed by feedback we receive from past attendees through the 6-month follow-up survey. This feedback indicates a strong need for advanced training as well as practice opportunities. Rather than develop an advanced iteration of the Institute to meet the need for continued training, which would be labor- and time-intensive, we could instead offer a series of free, specialized workshops that focus on discipline-specific guidelines, tools, and resources. This will provide advanced training while simultaneously expanding our instructor base by creating more instruction opportunities. Additionally, best-practice guidelines and technology are continuously evolving; therefore, training will need to remain up-to-date to reflect rapid developments in research methods.
The ability for librarians to support ES is reliant on administrative support. In a recent study, Lê et al. (2024a) highlighted that while 54.4% of librarian respondents felt that their administrators were supportive of their participation in ES, 3.4% felt that their administration discouraged participation, and 14.3% of respondents were unsure of how their administration felt about this. A subsequent report by the same team (Lê et al. 2024b) found that librarians were less likely to report training or travel support for ES skill development when their administration expressed negative views. Harnessing support from administrators is key for developing skills in this emerging area and for ensuring subsequent opportunities to practice and participate once the skills are developed.
We began ESI at a time when evidence synthesis training for librarians working outside of the health sciences was non-existent. Those few librarians who were versed in ES methods were working in relative isolation without adequate support or an active professional network. Despite this state of affairs, as well as the massive challenges posed by the pandemic, the Institute has been responsible for training over 500 librarians in evidence synthesis methodologies. It provides a library of free resources to aid in instruction and professional development, and it has helped foster a network that connects librarians across the United States and beyond so that no one has to engage in this work alone. Looking to the future, there is work to be done to maintain and grow the program we have built. In order to keep pace with the continual growth of evidence synthesis research and meet researchers’ evolving needs, we must also grow by training more instructors, expanding our training to cover emerging topics and new tools, and developing innovative instructional resources that are accessible to all learners. At the end of the IMLS grant period, ESI is positioned at the forefront of this emergent area of librarianship. With continued backing, the Institute can usher more librarians into its supportive community and continue to improve the quality of evidence synthesis research on a global scale.
Author Contributions
Megan Kocher: Conceptualization, Funding acquisition, Methodology, Project administration, Supervision, Writing - original draft, Writing - review & editing Amy Riegelman: Conceptualization, Funding acquisition, Supervision, Visualization, Writing - original draft, Writing - review & editing Matthew Kibbee: Writing - original draft, Writing - review & editing Molly Bostrom: Writing - original draft, Writing - review & editing Scott Marsalis: Writing - original draft, Writing - review & editing Sarah Young: Visualization, Writing - original draft, Writing - review & editing Zahra Premji: Writing - original draft Lindsay Bahureksa: Writing - original draft Natasha Bishop: Writing - original draft Gracian Chimwaza: Writing - original draft Cozette Comer: Writing - original draft Elizabeth DiGiovine: Writing - original draft
References
Aamodt, M., Huurdeman, H., & Strømme, H. (2019). Librarian co-authored systematic reviews are associated with lower risk of bias compared to systematic reviews with acknowledgement of librarians or no participation by librarians. Evidence Based Library and Information Practice, 14(4), 103–127. https://doi.org/10.18438/eblip29601
Aloe, A. M., Dewidar, O., Hennessy, E. A., Pigott, T., Stewart, G., Welch, V., Wilson, D. B., & Campbell MECCIR Working Group. (2024). Campbell standards: modernizing Campbell's methodologic expectations for Campbell Collaboration Intervention Reviews (MECCIR). Campbell Systematic Reviews, 20(4), e1445. https://doi.org/10.1002/cl2.1445
Bahureksa, L. (2024, October 18). Evidence synthesis in my discipline: yes, no, maybe so? [Conference presentation]. Upstate NY Science Librarians Meeting 2024, Cortland, NY, United States.
Boden, C., Bolton, S., & Gerrard, A. (2025). Health sciences and beyond: An investigation into Canadian librarian participation in systematic reviews across disciplines. College & Research Libraries, 86(1), 79. https://doi.org/10.5860/crl.86.1.79
Chalmers, I., Hedges, L. V., & Cooper, H. (2002). A brief history of research synthesis. Evaluation & the Health Professions, 25(1), 12-37. https://doi.org/10.1177/0163278702025001003
Evidence synthesis articles. (n.d.). Zotero. https://www.zotero.org/groups/2418750/evidence_synthesis_articles/library
Evidence Synthesis Institute. (n.d.). Evidence synthesis institute. Pressbooks. https://pressbooks.umn.edu/evidencesynthesisinstitute/
Evidence synthesis institute. (n.d.). University Libraries. https://z.umn.edu/esinstitute
Evidence synthesis institute Canada. (2025, December 18). University of Victoria Libraries. https://libguides.uvic.ca/c.php?g=731009&p=5367754
Expertsearching. (n.d.). Medical Library Association. Retrieved February 14, 2025, from https://pss.mlanet.org/mailman/listinfo/expertsearching_pss.mlanet.org
Ghezzi‐Kopel, K., Ault, J., Chimwaza, G., Diekmann, F., Eldermire, E., Gathoni, N., Kelly, J., Kinengyere, A. A., Kocher, M., Lwoga, E. T., Page, J., Young, S., & Porciello, J. (2022). Making the case for librarian expertise to support evidence synthesis for the sustainable development goals. Research Synthesis Methods, 13(1), 77–87. https://doi.org/10.1002/jrsm.1528
Information Training and Outreach Centre for Africa. (2025, May 5). Evidence synthesis training for African librarians. https://itoca.org/workshop/8
International Institute for Sustainable Development. (n.d.). CERES2030: Sustainable solutions to end hunger. https://www.iisd.org/projects/ceres2030-sustainable-solutions-end-hunger
Knowledge Synthesis Interest Group. (n.d.). CHLA-ABSC. Retrieved February 14, 2025, from https://ksig.chla-absc.ca/
Kocher, M., Kelly, J., & Riegelman, A. (2020). Advancing Systematic Review Practices beyond the Health Sciences: Training Librarians to Support Evidence Synthesis in Emerging Contexts (RE-246347-OLS-20). IMLS. https://hdl.handle.net/11299/269446
Kocher, M, Riegelman, A., Kibbee, M., Young, S., Bostrom, M., Marsalis, S., & Premji, Z. (2025). Evidence synthesis for librarians and information specialists [Online course]. Open Learning Initiative. https://oli.cmu.edu/courses/evidence-synthesis-for-librarians-and-information-specialists-o-f/
Koffel, J. B. (2015). Use of recommended search strategies in systematic reviews and the impact of librarian involvement: A cross-sectional survey of recent authors. PLOS ONE, 10(5), e0125931. https://doi.org/10.1371/journal.pone.0125931
Kogut, A., Ramirez, D., & Foster, M. J. (2022). Systematic review training model for education librarians: A case study. New Review of Academic Librarianship, 28(2), 205–226. https://doi.org/10.1080/13614533.2020.1784761
Lafferty, M., Premji, Z., Herold, P., Kocher, M., & Marsalis, S. (2024). Evidence synthesis instructional offerings in library and information science programs. Evidence Based Library and Information Practice, 19(4), 18–38. https://doi.org/10.18438/eblip30554
Lê, M. L., Neilson, C. J., & Winkler, J. (2024a). Benchmarking librarian support of systematic reviews in the sciences, humanities, and social sciences. College & Research Libraries, 85(4), 606. https://doi.org/10.5860/crl.85.4.606
Lê, M. L., Winkler, J., & Neilson, C. J. (2024b). Training needs and preferences for librarians supporting systematic reviews in the sciences, humanities, and social sciences. College & Research Libraries, 85(7), 978. https://doi.org/10.5860/crl.85.7.978
Medical Library Association. (n.d.). Systematic review services specialization. https://www.mlanet.org/Specializations/systematic-review-services-specialization/
Meert, D., Torabi, N., & Costella, J. (2017). Impact of librarians on reporting of the literature searching component of pediatric systematic reviews. Journal of the Medical Library Association, 104(4). https://doi.org/10.5195/jmla.2016.139
Pawliuk, C., Cheng, S., Zheng, A., & Siden, H. (2024). Librarian involvement in systematic reviews was associated with higher quality of reported search methods: A cross-sectional survey. Journal of Clinical Epidemiology, 166, 111237. https://doi.org/10.1016/j.jclinepi.2023.111237
Premji, Z., Splenda, R., & Young, S. (2022). An exploration of business librarian participation in knowledge synthesis reviews. College & Research Libraries, 83(2). https://doi.org/10.5860/crl.83.2.314
Rethlefsen, M. L., Farrell, A. M., Osterhaus Trzasko, L. C., & Brigham, T. J. (2015). Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews. Journal of Clinical Epidemiology, 68(6), 617–626. https://doi.org/10.1016/j.jclinepi.2014.11.025
Slebodnik, M., Cahoy, E. S., & Jacobsen, A. L. (2022). Evidence synthesis: Coming soon to a library near you? portal: Libraries and the Academy, 22(2), 273–280. https://doi.org/10.1353/pla.2022.0016
umnLibraries. (n.d.). Evidence synthesis institute [YouTube playlist]. Retrieved January 16, 2026, from https://www.youtube.com/playlist?list=PLsqxtHlAYQnYM0A_aLNa9GeWMY_E2ijTP
University of Michigan Library. (n.d.). Systematic reviews workshop. https://www.lib.umich.edu/research-and-scholarship/library-workshops-and-credit-courses/systematic-reviews-workshop