Evidence Based Library and Information Practice
type: channel
publisher: University of Alberta Library
language: en-US
publication name: Evidence Based Library and Information Practice
issn: 1715-720X
copyright: The Creative Commons-Attribution-Noncommercial-Share Alike License 4.0 International applies to all works published by Evidence Based Library and Information Practice. Authors will retain copyright of the work.
items: Publishing Assessment Projects as Research
items: Examining the Meaning and Methodological Characteristics of the Systematized Review Label: A Scoping Review
items: Developing an OPEN Framework for Asking EBLIP Questions in Open Education
items: Engagement Practices of Canadian Academic Libraries for the Advancement of Student Voices
items: The Evidence Synthesis Institute: Building a High Impact Training Program and Supportive Community for an Emergent Area of Librarianship
items: Instructing Practicing Physicians in Their Continuing Professional Development: A Survey of Canadian Health Sciences Librarians
items: Pragmatic Human-Centred Design in Action: Assessing the Tone and Usability of the Behaviours in Dementia Toolkit Website
items: The Knowledge, Attitudes, and Practices of Academic Librarians Toward Open Educational Resources: Developing and Validating the OpenEd-LibKAP Scale
items: Call for Applications: Associate Editor (Research Articles) for Evidence Based Library and Information Practice
items: Call for Applications: Copyeditors for Evidence Based Library and Information Practice
items: Discrepancies Found in Librarian and Patron Perceptions of Successful Virtual Chat Interactions
items: Scholarly Material Access Barriers for Reference and Instruction Librarians Seeking Non-English Research
items: The Illusion of Care: Limited and Superficial Caring Discourse in Bilingual Public Library Interactions
items: Improving Chat Reference Referrals Through Enhanced Communication, Empathetic Protocols, and Evidence Based Training Practices
items: Popular GenAI Chatbots Vary in Capabilities to Answer Academic Reference Questions

Publishing Assessment Projects as Research
type: item
creator: Ann Medaille
rights: Copyright (c) 2026 Ann Medaille https://creativecommons.org/licenses/by-nc-sa/4.0
license: https://creativecommons.org/licenses/by-nc-sa/4.0
date: 2026-03-16
publication date: 2026-03-16
volume: 21
number: 1
starting page: 1
ending page: 3
doi: 10.18438/eblip31053

Examining the Meaning and Methodological Characteristics of the Systematized Review Label: A Scoping Review
type: item
description: Objective – In 2009, a typology by Grant and Booth introduced the concept of a systematized review in which authors (typically students) selectively employ various elements of the systematic review process. As academic librarians who help research teams to select and conduct a variety of evidence synthesis types, we have fielded inquiries from teams interested in conducting systematized reviews for publication. While the typology is widely referenced, we were aware of no formal methodological guide to a systematized review type. The objective of this scoping review is to identify and describe the extent of published systematized reviews, and to 1) identify and collate, where available, sources used for the conceptualization and conduct of the systematized reviews, 2) determine if explanations provided were based on constraints, and 3) describe common methodological characteristics. Methods – Articles titled as a systematized review that attempted the collocation and synthesis of literature and included an adequate description of their methodology were included. We searched on September 1, 2023, the following five sources: Google Scholar, Lens.org, Web of Science Core Collection, Scopus, and MEDLINE. We performed screening and data extraction in duplicate. Data extraction elements included common methodological characteristics relating to various steps of the evidence synthesis process. Descriptive, aggregate statistics, and categorization of reasons for selecting the systematized review type were the primary analysis for this review. Results – Our review found 171 systematized reviews that met inclusion criteria. These were published between 2013 and 2023, and the number has increased each year. Resources or constraints were mentioned in only 32 included reviews, and only 19 had a single author. The methodological attributes of published systematized reviews vary significantly. A small number (15) of reviews searched only one database or source, while the majority s…
creator: Leyla Cabugos
creator: Zahra Premji
rights: Copyright (c) 2026 Leyla Cabugos, Zahra Premji https://creativecommons.org/licenses/by-nc-sa/4.0
license: https://creativecommons.org/licenses/by-nc-sa/4.0
date: 2026-03-16
publication date: 2026-03-16
volume: 21
number: 1
starting page: 167
ending page: 193
doi: 10.18438/eblip30757

Developing an OPEN Framework for Asking EBLIP Questions in Open Education
type: item
description: Objective – This paper proposes a novel framework for asking questions in evidence based library and information practice (EBLIP) called OPEN (Objective, Purpose, Evidence, and Narrative). It responds to the question: How can a framework for asking EBLIP questions be developed and applied to open educational practices (OEP)? Methods – Using a single-case study design, the framework was developed using an inductive approach throughout a three-year collaborative project to create a data dashboard. The project was documented, and these qualitative project artefacts were analyzed. The analysis was used to develop the framework and determine its value. Results – Arising concurrently from the data dashboard project, the OPEN framework comprises four elements - Objective: What do we need to know?; Purpose: Why do we need to know this?; Evidence: What evidence do I have or need?; and Narrative: How will I communicate this evidence? These elements guide library and information professionals to define what they need to know, collect, and communicate to make evidence based decisions. The case study demonstrates how the framework can be applied to OEP. Conclusion – Existing evidence based practice (EBP) frameworks developed for clinical environments are difficult to apply to the wide range of EBLIP initiatives, which are often more exploratory, or more immediate in their evidence needs. UniSQ Library's collaborative data dashboard project highlighted the need to develop a flexible framework that diverse stakeholders, including OEP practitioners, with differing levels of experience in evidence based practice, could apply. This research found that the collaborative and reflective nature of the project was instrumental in developing both a useful data dashboard to empower authors to tell their own data stories, and a new framework that contributes to the future of EBLIP and enhances the ability of OEP practitioners to meaningfully engage with EBP.
creator: Emilia C. Bell
creator: Adrian Stagg
rights: Copyright (c) 2026 Emilia C. Bell, Adrian Stagg https://creativecommons.org/licenses/by-nc-sa/4.0
license: https://creativecommons.org/licenses/by-nc-sa/4.0
date: 2026-03-16
publication date: 2026-03-16
volume: 21
number: 1
starting page: 4
ending page: 21
doi: 10.18438/eblip30867

Engagement Practices of Canadian Academic Libraries for the Advancement of Student Voices
type: item
description: Objective – Student engagement is an integral part of the mission and function of academic libraries, with powerful potential to impact decision-making across various facets of governance, operations, services, and space design. Despite its importance, however, “student engagement” is a broad and highly contested term, even within the domain of libraries. While recent efforts have been made to categorize the concept of “student engagement” in this context and numerous case studies exist to highlight individual libraries’ efforts, successes, and challenges, there is a lack of existing studies involving a cohesive exploration across Canadian academic libraries. In response to this gap, this research details a national-level investigation of how academic libraries in Canada are approaching student engagement, with specific attention towards use of student voices as a feedback mechanism for decision-making. Methods – Using a combination of a national virtual survey and semi-structured interviews/focus groups with library employees, this study explores specific engagement strategies and processes used by academic libraries, as well as the perceived challenges and benefits of this work. Results – Sample findings indicate that, while library staff are generally enthusiastic about engagement and voice-seeking work, limited time, resources, and complex organizational cultures can limit activities and alienate students. Despite challenges, participants highlighted numerous benefits, including increased proximity and relevance to students, upholding of the key mission and goals of academic libraries, enhanced quality assessment and improvement of library operations through evidence based decision-making, and bolstered ability to pursue authentic equity, diversity, inclusion, and accessibility (EDIA) commitments. Feedback mechanisms, student employment, and teaching and learning interactions were the most reported strategies for engagement, though opportunities for other engage…
creator: Sarah Guay
rights: Copyright (c) 2026 Sarah Guay https://creativecommons.org/licenses/by-nc-sa/4.0
license: https://creativecommons.org/licenses/by-nc-sa/4.0
date: 2026-03-16
publication date: 2026-03-16
volume: 21
number: 1
starting page: 22
ending page: 56
doi: 10.18438/eblip30735

The Evidence Synthesis Institute: Building a High Impact Training Program and Supportive Community for an Emergent Area of Librarianship
type: item
description: Objective – Seeing an absence of training opportunities for librarians supporting evidence synthesis outside of the health sciences, a collaborative team of librarians sought grant funding, developed a curriculum, and facilitated community building to provide this training. Methods – The present case study describes the identification of a training gap, multi-faceted efforts made to fill that gap, and evaluation via pretests and post-tests to continuously improve the program. Results – The Evidence Synthesis Institute (ESI) was launched in 2020. An application process is used to select the 50-person ESI cohort, with an average acceptance rate of 37%. To date, over 500 individuals have attended the Evidence Synthesis Institutes. Pre- and post-institute surveys show consistent gains in confidence to perform a variety of evidence synthesis skills. Additionally, we developed a self-paced online course which has over 750 enrolled learners with 118 having completed at least 90% of the course. Conclusion – This case study provides a model for designing, implementing, and assessing a training program and community of practice for an emergent skillset in academic librarianship. As shown by the program’s acceptance rate, demand remains high for participation in evidence synthesis training. Future programming could build on past success, such as continuing to offer synchronous virtual Institutes, training additional instructors, and expanding the asynchronous online course to include more content and summative assessments.
creator: Megan Kocher
creator: Amy Riegelman
creator: Matthew Kibbee
creator: Molly Bostrom
creator: Scott Marsalis
creator: Sarah Young
creator: Zahra Premji
creator: Lindsay Bahureksa
creator: Natasha Bishop
creator: Gracian Chimwaza
creator: Cozette Comer
creator: Elizabeth DiGiovine
rights: Copyright (c) 2026 Megan Kocher, Amy Riegelman, Matthew Kibbee, Molly Bostrom, Scott Marsalis, Sarah Young, Zahra Premji, Lindsay Bahureksa, Natasha Bishop, Gracian Chimwaza, Cozette Comer, Elizabeth DiGiovine https://creativecommons.org/licenses/by-nc-sa/4.0
license: https://creativecommons.org/licenses/by-nc-sa/4.0
date: 2026-03-16
publication date: 2026-03-16
volume: 21
number: 1
starting page: 57
ending page: 74
doi: 10.18438/eblip30736

Instructing Practicing Physicians in Their Continuing Professional Development: A Survey of Canadian Health Sciences Librarians
type: item
description: Objective – Practicing physicians must engage in continuous learning throughout their careers, and in Canada this learning is guided by frameworks that include information retrieval and scholarship. Health sciences librarians (HSLs) are well equipped to support knowledge growth in this area through their demonstrated instruction capabilities. Yet the literature depicting librarian instruction of practicing physicians within the Canadian context is lacking, and an understanding of what characterizes their teaching to this learner group is needed. Methods – Using descriptive study design, a bilingual, electronic survey was distributed across two professional listservs for HSLs in North America in May 2024. Once the survey closed, data were cleaned and tabulated using Microsoft Excel and responses to open-ended questions were reviewed. Results – The survey engaged 21 respondents, and after data were cleaned, 15 were left for inclusion and analysis. Most worked in a hospital setting when teaching practicing physicians, and online synchronous delivery methods were most common, followed closely by in-person synchronous. A variety of topics were reportedly taught, and the use of formal objectives and feedback processes was common. In open-ended responses, participants suggested they may be interested in teaching practicing physicians about artificial intelligence. Conclusion – This study’s sample size is too small to offer definitive conclusions. However, it provides insight into what and how HSLs in Canada are teaching practicing physicians, and suggests missing components in planning instruction for this learner group. HSLs can provide value to Canadian physician organizations looking to bolster their CPD offerings and should advocate for themselves within their provincial and national networks.
creator: Jackie Phinney
creator: Alison Manley
rights: Copyright (c) 2026 Jackie Phinney, Alison Manley https://creativecommons.org/licenses/by-nc-sa/4.0
license: https://creativecommons.org/licenses/by-nc-sa/4.0
date: 2026-03-16
publication date: 2026-03-16
volume: 21
number: 1
starting page: 75
ending page: 97
doi: 10.18438/eblip30830

Pragmatic Human-Centred Design in Action: Assessing the Tone and Usability of the Behaviours in Dementia Toolkit Website
type: item
description: Objective – The Behaviours in Dementia Toolkit is an online library to support dementia-related mood and behaviour changes. It is intended to be used by two key caregiving audiences: 1) formal, professional care partners, and 2) informal care partners. The purpose of this study was to assess the content tone and usability of the website to meet the needs of its two user groups. Methods – The multi-method design included electronic surveys and interviews. Content tone was assessed using word reaction and highlighter tests. Task-specific usability was assessed with the single-ease question (SEQ), time on task, and completion rate. The System Usability Scale (SUS) measured overall usability before and after design changes based on initial results. Results – Content tone was friendly, trustworthy, desirable, and clear, but informal care partners found medical jargon confusing. Task-specific usability was generally acceptable. Time one SUS score for formal care partners was assessed at a letter grade A, but for informal care partners was a C. Content tone and usability testing drove critical changes to the library. Time two SUS scores subsequently equalized for the user groups to an A- for formal care partners and a B for informal care partners. Divergent requirements for formal and informal care partners were identified and addressed using pragmatic human-centred design methods. Conclusion – Jargon, information overload, and search flow require careful attention in an online health information library. Digital libraries and other projects facilitating access to health information seeking groups may find these usability methods useful to consider at the outset of their project planning given the actionable insights they can provide. These techniques are suitable for projects with a limited budget and tight timeframe.
creator: Nick Ubels
creator: Lauren Albrecht
creator: Woon Chee Koh
creator: David Conn
rights: Copyright (c) 2026 Nick Ubels, Lauren Albrecht, Woon Chee Koh, David Conn https://creativecommons.org/licenses/by-nc-sa/4.0
license: https://creativecommons.org/licenses/by-nc-sa/4.0
date: 2026-03-16
publication date: 2026-03-16
volume: 21
number: 1
starting page: 98
ending page: 140
doi: 10.18438/eblip30754

The Knowledge, Attitudes, and Practices of Academic Librarians Toward Open Educational Resources: Developing and Validating the OpenEd-LibKAP Scale
type: item
description: Objective – The aims of the open education (OE) movement can be supported by academic librarians, although most librarians have not had formal training on open educational resources (OER). The objective of this study is to design and validate a survey instrument to measure the knowledge, attitudes, and practices (KAP) of academic librarians toward OER to be used in future research projects. Methods – The Open Education - Librarian Knowledge, Attitudes, and Practices (OpenEd-LibKAP) scale was developed by experienced academic librarians and assessed for content validity by OER subject experts. A pilot study was conducted to assess internal consistency, and a second round of the survey was administered to an international group of current academic librarians. Exploratory factor analysis (EFA) was performed on the results. Results – The final instrument includes 50 questions, with 22 items in the knowledge domain, 16 items in the attitude domain, and 12 items in the practices domain. The KAP factors positively correlate in an expected manner, with a range from .404 to .591. The individual domains have high markers of reliability, implying a degree of confidence in our findings and future uses of the tool. Conclusion – The OpenEd-LibKAP Scale developed through this study can be used by library administrators, OER program administrators, librarian researchers, and OER researchers to more accurately measure and assess academic librarian OER competencies, beliefs, and behaviours related to OER.
creator: Riley Richards
creator: Jennifer Monnin
creator: Kristin Whitman
rights: Copyright (c) 2026 Riley Richards, Jennifer Monnin, Kristin Whitman https://creativecommons.org/licenses/by-nc-sa/4.0
license: https://creativecommons.org/licenses/by-nc-sa/4.0
date: 2026-03-16
publication date: 2026-03-16
volume: 21
number: 1
starting page: 141
ending page: 166
doi: 10.18438/eblip30707

Call for Applications: Associate Editor (Research Articles) for Evidence Based Library and Information Practice
type: item
creator: Editorial Team
rights: Copyright (c) 2026 Editorial Team https://creativecommons.org/licenses/by-nc-sa/4.0
license: https://creativecommons.org/licenses/by-nc-sa/4.0
date: 2026-03-16
publication date: 2026-03-16
volume: 21
number: 1
starting page: 209
ending page: 210
doi: 10.18438/eblip31040

Call for Applications: Copyeditors for Evidence Based Library and Information Practice
type: item
creator: Editorial Team
rights: Copyright (c) 2026 Editorial Team https://creativecommons.org/licenses/by-nc-sa/4.0
license: https://creativecommons.org/licenses/by-nc-sa/4.0
date: 2026-03-16
publication date: 2026-03-16
volume: 21
number: 1
starting page: 211
ending page: 212
doi: 10.18438/eblip31034

Discrepancies Found in Librarian and Patron Perceptions of Successful Virtual Chat Interactions
type: item
description: A Review of: Owens, E., & Brooks, K. (2025). Comparison of librarian and patron ratings of synchronous chat interactions. College & Research Libraries, 86(4), 567–585. https://doi.org/10.5860/crl.86.4.567 Objective – To compare librarian and patron assessments of virtual chat interactions and identify trends in librarian and patron ratings. Design – Rubric-guided content analysis and scoring of LibChat transcripts. Setting – Two large academic libraries at two public universities in the United States. Subjects – 710 virtual chat transcripts of patron and library worker interactions. Methods – The researchers downloaded, blinded, and cleaned the data of LibChat transcripts that included patron ratings spanning a one-year period. A randomized sample of 360 transcripts from each institution were selected for analysis. Using an adapted rubric based on the RUSA Guidelines for Behavioral Performance of Reference and Information Service Providers, transcripts were divided evenly between two researchers for scoring. Transcripts were scored on a scale of 1 to 3 (as Beginning, Developing, or Accomplished) in the areas of Listening, Interest, Searching, and Follow Up for a maximum of 12 points. The researchers’ scores were compared to the patron ratings using descriptive statistics and Pearson’s correlation tests. Transcripts were also evaluated qualitatively to identify themes in patron ratings. Main Results – The study found that patrons and librarians were generally satisfied with chat interactions, with patrons giving an average rating of 3.8 (out of 4) and librarians scoring transcripts an average of 9.5 (out of 12). Overall, patrons were more likely to consider a chat interaction Accomplished (86.3%) when compared to librarians (56.2%), whereas librarians were more likely to consider a chat interaction Beginning (8.7%) than patrons (1.5%). The few cases (n=8) where the patron gave a low rating and the librarian gave a high score were instances in which the librarian met …
creator: Julia Hayes
rights: Copyright (c) 2026 Julia Hayes https://creativecommons.org/licenses/by-nc-sa/4.0
license: https://creativecommons.org/licenses/by-nc-sa/4.0
date: 2026-03-16
publication date: 2026-03-16
volume: 21
number: 1
starting page: 194
ending page: 196
doi: 10.18438/eblip30937

Scholarly Material Access Barriers for Reference and Instruction Librarians Seeking Non-English Research
type: item
description: A Review of: Absher, L. U. & Desilets, M. (2024). English as the scholarly language: Diversity, equity, and inclusion implications for academic reference and instruction librarians. The Library Quarterly, 94(3), 296–315. https://doi.org/10.1086/730464 Objective – To examine the use of non-English resources in academic libraries, specifically in reference and instruction. Research questions include: (1) Do academic librarians in the United States and Canada suggest non-English language resources in their reference and instruction practice, and to what extent? (2) What factors or cues prompt an offer of non-English language resources? Design – Literature review and subsequent survey. Setting – Online. Subjects – Academic librarians from Canada and the United States. Methods – A 32-question survey including demographics, language, educational background, collection development, reference activities, instruction, and librarian attitudes was disseminated in the winter of 2020-2021. Main Results – The survey respondents were primarily native English speakers. Regarding collection development, more than 75% of 131 respondents indicated that they purchase or acquire materials in languages other than English. When asked about areas of collection development, the highest areas of concentration were social sciences and humanities. More than 75% of participants stated comfortability in responding to reference questions regarding non-English materials; however, only 10% indicated they would seek out non-English materials without a specific request. Conclusion – Based on the survey responses, academic librarians use non-English resources reactively, primarily offering them only due to specific request. There may also be a disconnect between the perceived likelihood of patrons using these resources proactively and the actual practice of doing so.
creator: Hilary Jasmin
rights: Copyright (c) 2026 Hilary Jasmin https://creativecommons.org/licenses/by-nc-sa/4.0
license: https://creativecommons.org/licenses/by-nc-sa/4.0
date: 2026-03-16
publication date: 2026-03-16
volume: 21
number: 1
starting page: 197
ending page: 199
doi: 10.18438/eblip30938

The Illusion of Care: Limited and Superficial Caring Discourse in Bilingual Public Library Interactions
type: item
description: A Review of: Frye, J., & Hasler-Barker, M. (2024). “Lady can talk forever...”: Exploring caring discourse in bilingual librarianship. Library & Information Science Research, 46(2), 1–9. https://doi.org/10.1016/j.lisr.2024.101301 Objective – To investigate how bilingual reference librarians in a public library express care in their interactions with community members, and to examine whether and how librarians’ linguistic choices reflect authentic caring discourse in multilingual contexts. Design – A case study of bilingual reference desk transactions. Setting – Online. Subjects – A public library in a U.S.-Mexico border town. Methods – Members of the research team digitally recorded approximately 20 hours of reference desk interactions over one week and collected extensive field notes. Bilingual transcribers produced full transcripts, categorized by language use (Spanish-only, English-only, bilingual). The research team examined the transcripts under the guidance of the critical discourse analysis methodology. The researchers used Fairclough’s three-dimensional framework (description, interpretation, and explanation) to summarize interactions, examine linguistic features, and analyze expressions of care in relation to social and cultural contexts. Coding of the transcripts was refined through feminist research practices, ensuring attentiveness, validation, and dialogue across researcher perspectives. Main Results – Caring discourse was infrequent in bilingual reference transactions. Out of 20 bilingual interactions, only 3 contained explicit expressions of care. The caring discourse was mostly brief, primarily delivered in English regardless of the community member’s language choice and often tied to emotional disclosures rather than the initial informational requests. The researchers identified three main categories of caring discourse: commiseration, soothing assurance, and expressions of condolence. These were supported by five conversational devices: interjection…
creator: Lili Luo
rights: Copyright (c) 2026 Lili Luo https://creativecommons.org/licenses/by-nc-sa/4.0
license: https://creativecommons.org/licenses/by-nc-sa/4.0
date: 2026-03-16
publication date: 2026-03-16
volume: 21
number: 1
starting page: 200
ending page: 202
doi: 10.18438/eblip30884

Improving Chat Reference Referrals Through Enhanced Communication, Empathetic Protocols, and Evidence Based Training Practices
type: item
description: A Review of: Saulnier Lange, J., Johnson, C., & Martin, P. (2024). Service, interrupted: Analyzing chat reference referrals. The Reference Librarian, 65(1–2), 34–58. https://doi.org/10.1080/02763877.2024.2304360 Objective – To analyze service interruptions in chat reference interactions to determine best practices for chat reference delivery and training. Design – Mixed methods analysis of chat reference transcripts using grounded theory. Setting – One large university library in the United States. Subjects – Three hundred chat reference transcripts from the period during and after COVID-19 pandemic closure, that included an identified service interruption such as a referral, deferral, or missed referral. Methods – Researchers analyzed a random sample of relevant chat reference transcripts from May 2020 to June 2022, using a standardized data collection form based on observed transcript patterns and established coding schemes from relevant studies. The analysis considered the following factors: Query type, whether a referral was warranted, frequency of disconnected chats, operator actions, referral recipients, reference strategies employed, knowledge gaps, customer service level, and patron satisfaction level. Main Results – Most service disruptions involved referrals, with reference, library account, and item requests being the most common query types. Disconnected chat rates were lower than expected. In most cases, operators either directly contacted the referred person/unit or provided contact information. They used reference strategies such as asking clarifying questions, attempting to solve the patron’s issue, ensuring correct patron contact information, and inviting additional questions before closing the conversation. Most interactions on the patron satisfaction level were labelled as “polite” (choices were: "active dissatisfaction," "polite," and "extreme gratitude/happiness"), particularly when there was an attempt by the operator to resolve the issue. Cust…
creator: Lisa Shamchuk
rights: Copyright (c) 2026 Lisa Shamchuk https://creativecommons.org/licenses/by-nc-sa/4.0
license: https://creativecommons.org/licenses/by-nc-sa/4.0
date: 2026-03-16
publication date: 2026-03-16
volume: 21
number: 1
starting page: 203
ending page: 205
doi: 10.18438/eblip30912

Popular GenAI Chatbots Vary in Capabilities to Answer Academic Reference Questions
type: item
description: A Review of: Whitfield, S., & Yang, S. Q. (2025). Evaluating AI language models for reference services: A comparative study of ChatGPT, Gemini, and Copilot. Internet Reference Services Quarterly, 29(2), 153-167. https://doi.org/10.1080/10875301.2025.2478861 Objective – To assess and compare the quality of responses to reference questions by popular generative artificial intelligent (GenAI) chatbots. Design – Content analysis. Setting – Web browser platforms of four GenAI chatbots. Subjects – Responses from ChatGPT 3.5, Chat GPT 4.0, Gemini, and Copilot to a set of 28 chat reference questions submitted by Rider University Library patrons between July 1, 2023, and May 14, 2024. Methods – Transcripts of 112 responses from the four chatbots were evaluated using a content analysis scheme adapted from a previous ChatGPT study (Yang & Mason, 2024). Both researchers independently rated every response using a 10-point scale on four categories, accuracy, relevance, friendliness, and instructiveness, and analyzed the results using inferential statistics. Main Results – Responses from Gemini received the highest total score (592 out of a possible 1,120 points), followed by ChatGPT 4.0 (542), ChatGPT 3.5 (502), and Copilot (433). However, every chatbot fluctuated greatly in their performance in accuracy and relevance. A single-factor ANOVA test also found statistically significant differences between the quality of the GenAI chatbots’ responses in relevance and friendliness, with Gemini and Copilot performing the best in these categories respectively. There were no statistically significant differences between the chatbots’ performances in accuracy or instructiveness, although Gemini held the highest mean score for instructiveness. Conclusion – The researchers concluded that popular GenAI chatbots should be used to supplement, not replace, the work of reference and instruction librarians, and noted the potentials for training GenAI to address basic or local, library-specific FAQ…
creator: Lisa Shen
rights: Copyright (c) 2026 Lisa Shen https://creativecommons.org/licenses/by-nc-sa/4.0
license: https://creativecommons.org/licenses/by-nc-sa/4.0
date: 2026-03-16
publication date: 2026-03-16
volume: 21
number: 1
starting page: 206
ending page: 208
doi: 10.18438/eblip30949