Evidence Summary
A Review of:
Rodriguez, S., & Mune, C. (2022). Uncoding library chatbots: Deploying a new virtual
reference tool at the San Jose State University Library. Reference Services
Review, 50(3), 392-405. https://doi.org/10.1108/RSR-05-2022-0020
Reviewed by:
Matthew Chase
Instructional Services
Librarian
Cuyamaca College
Rancho San Diego,
California, United States of America
Email: matthew.chase@gcccd.edu
Received: 1 Mar. 2024 Accepted: 23 Apr. 2024
2024 Chase.
This is an Open Access article distributed under the terms of the Creative
Commons‐Attribution‐Noncommercial‐Share Alike License 4.0
International (http://creativecommons.org/licenses/by-nc-sa/4.0/),
which permits unrestricted use, distribution, and reproduction in any medium,
provided the original work is properly attributed, not used for commercial
purposes, and, if transformed, the resulting work is redistributed under the
same or similar license to this one.
DOI: 10.18438/eblip30523
Objective – To describe the
development of an artificial intelligence (AI) chatbot to support virtual
reference services at an academic library.
Design – Case study.
Setting – A public
university library in the United States.
Subjects – 1,682
chatbot-user interactions.
Methods – A university
librarian and two graduate student interns researched and developed an AI
chatbot to meet virtual reference needs. Developed using chatbot development
software, Dialogflow, the chatbot was populated with
questions, keywords, and other training phrases entered during user inquiries,
text-based responses to inquiries, and intents (i.e., programmed mappings
between user inquiries and chatbot responses). The chatbot utilized natural
language processing and AI training for basic circulation and reference
questions, and included interactive elements and embeddable widgets supported
by Kommunicate (i.e., a bot support platform for chat
widgets). The chatbot was enabled after live reference hours were over. User
interactions with the chatbot were collected across 18 months since its launch.
The authors used analytics from Kommunicate and Dialogflow to examine user interactions.
Main Results – User
interactions increased gradually since the launch of the chatbot. The chatbot
logged approximately 44 monthly interactions during the spring 2021 term, which
increased to approximately 137 monthly interactions during the spring 2022
term. The authors identified the most common reasons for users to engage the
chatbot, using the chatbot’s triggered intents from user inquiries. These
reasons included information about hours for the library building and live
reference services, finding library resources (e.g., peer-reviewed articles,
books), getting help from a librarian, locating databases and research guides,
information about borrowing library items (e.g., laptops, books), and reporting
issues with library resources.
Conclusion – Libraries can
successfully develop and train AI chatbots with minimal technical expertise and
resources. The authors offered user experience considerations from their
experience with the project, including editing library FAQs to be concise and
easy to understand, testing and ensuring chatbot text and elements are
accessible, and continuous maintenance of chatbot content. Kommunicate,
Dialogflow, Google Analytics, and Crazy Egg (i.e., a
web usage analytics tool) could not provide more in-depth user data (e.g., user
clicks, scroll maps, heat maps), with plans to further explore other usage
analysis software to collect the data. The authors noted that only 10% of users
engaged the chatbot beyond the initial welcome prompt, requiring more research
and user testing on how to facilitate user engagement.
There is growing LIS literature on the applications of
library chatbots in library virtual reference services. Chatbots can help
reduce user stress and anxiety as well as ease the workload of common and
routine questions for library staff (McNeal & Newyear, 2013; Mckie & Narayan, 2019). However, AI is still an
emerging and challenging trend for libraries, especially when it comes to
usage, technical expertise, and resources. Despite the opportunity to automate
library services, human management in a chatbot’s development, training, and
maintenance are essential (Kane, 2019). In this study, the authors offered an
interesting approach to developing a chatbot for supporting a library’s virtual
reference services without requiring specialized experience or abundant
resources.
The Glynn (2006) critical appraisal tool was applied
to evaluate this study. While the authors clearly described the data collection
methods, the authors did not address the possible subjectivity with the
chatbot’s interpretation of user inquiries based on which intents were
triggered. The authors created the chatbot with the aim to assist students
after live chat reference hours; however, they did not clearly identify
inclusion and exclusion criteria, namely the user’s affiliation with the
university (e.g., student, faculty, alum). The authors noted that the chatbot
collects data from suggestion chips (i.e., predefined options for users to
select from an automated prompt), which can offer more in-depth analysis of
user satisfaction. This study did not analyze the suggestion chip data,
although the authors may plan to investigate in the future. Although the
authors did not evaluate the results of user interactions with the chatbot, the
study serves as a proof of concept regarding the development of a virtual
reference AI chatbot at an academic library.
The authors addressed that one major area for
additional research is user engagement with the chatbot. With only 10% of users
engaging with the chatbot beyond an initial response, the authors noted the
necessity to explore ways to understand user perceptions of chatbot services,
cultivate users’ assurance of the chatbot’s ability to meet their needs, and
build the chatbot’s natural language capabilities to deliver targeted content
to users.
This study provided insights into how academic
libraries can develop a chatbot with limited training, technical knowledge, and
resources. The authors clearly outlined methods and practices for other
libraries of nearly any type to incorporate chatbots in their virtual reference
services. Successful chatbot developments could create a precedent for other
libraries to follow, bridging the potential gaps and costs with chatbot
adoption such as lack of technological familiarity among library staff (Guy et
al., 2023). This study provided adaptable guidelines and considerations for
other libraries to create and customize their own chatbots, making it possible
for follow-up research across multiple institutions to investigate and apply
the potential benefits and challenges of chatbot reference services.
Glynn, L. (2006). A critical appraisal tool for
library and information research. Library
Hi Tech, 24(3), 387-399. http://dx.doi.org/10.1108/07378830610692154
Guy, J., Rival, P. R., Lewis, C. J., & Groome, K.
(2023). Reference chatbots in Canadian academic libraries. Information Technology and Libraries, 42(4). https://doi.org/10.5860/ital.v42i4.16511
Kane, D. (2019). Creating, managing and analyzing an
academic library chatbot. BiD: Textos Universitaris de Biblioteconomia i Documentació,
43(2019). https://bid.ub.edu/en/43/kane.htm
Mckie, I. A. S.,
& Narayan, B. (2019). Enhancing the academic library experience with
chatbots: An exploration of research and implications for practice. Journal of the Australian Library and
Information Association, 68(3),
268-277. https://doi.org/10.1080/24750158.2019.1611694
McNeal, M. L., & Newyear, D. (2013). Chapter 1: Introducing chatbots
in libraries. Library Technology Reports,
49(8), 5-10. https://www.journals.ala.org/index.php/ltr/article/view/4504/5281
Rodriguez, S., & Mune, C. (2022). Uncoding
library chatbots: Deploying a new virtual reference tool at the San Jose State
University Library. Reference Services
Review, 50(3), 392-405. https://doi.org/10.1108/RSR-05-2022-0020