Evidence Summary
Students’ Perspective of the
Advantages and Disadvantages of ChatGPT Compared to Reference Librarians
A Review of:
Adetayo, A. J. (2023). ChatGPT and librarians for reference
consultations. Internet Reference Services Quarterly, 27(3),
131–147. https://doi.org/10.1080/10875301.2023.2203681
Reviewed by:
Kathy Grams
Associate Professor of
Pharmacy Practice
Massachusetts College of
Pharmacy and Health Sciences
Boston, Massachusetts,
United States of America
Email: kathy.grams@mcphs.edu
Received: 18
Feb. 2024 Accepted: 3 Apr. 2024
2024 Grams.
This is an Open Access article distributed under the terms of the Creative
Commons‐Attribution‐Noncommercial‐Share Alike License 4.0
International (http://creativecommons.org/licenses/by-nc-sa/4.0/),
which permits unrestricted use, distribution, and reproduction in any medium,
provided the original work is properly attributed, not used for commercial
purposes, and, if transformed, the resulting work is redistributed under the
same or similar license to this one.
Abstract
Objective – To investigate students’ use of ChatGPT and its
potential advantages and disadvantages compared to reference librarians at a
university library.
Design – Survey research.
Setting – A university library in Nigeria.
Subjects – Students familiar with ChatGPT (n=54) who were
enrolled in a library users’ education course.
Methods – A survey was conducted in a sample of undergraduate
students enrolled in a library users’ education course, who had previously used
ChatGPT. Participants were asked questions based on six categories that
reflected frequency of use, types of inquiries, frequency of reference
consultations, desire to consult reference librarians despite the availability
of ChatGPT, and potential advantages and disadvantages of ChatGPT compared to
reference librarians. A 4-point Likert scale was used to measure the responses
from often to never, strongly agree to strongly disagree, and rarely to
frequently.
Main Results – The sample of students who participated (n=54) were
a diverse group whose age varied from below 20 (35.2%) to above 30 years
(31.5%) and represented a variety of fields of study, such as engineering,
business and social sciences, arts, law, sciences, basic and medical sciences.
Regarding frequency of use, the author reported that 40.7% of participants
occasionally used ChatGPT, and 26.1% and 16.7% used it frequently or very
frequently, respectively. Of the five options that represented types of
inquiries (religious, political, academic, entertainment, and work), academic
and work-related inquiries were topics most often searched in ChatGPT.
Participants indicated that they consulted reference librarians occasionally
(40.8%), frequently (37%), or rarely (22.2%). Most students (87%) would
continue to consult reference librarians despite the availability of ChatGPT.
For questions that compared ChatGPT to reference librarians, four options were
provided to describe potential advantages and four options were provided to
describe potential disadvantages. Most students agreed or strongly agreed that
ChatGPT is more user friendly (83.4%), that it includes a broad knowledge base
(90.7%), is easily accessible (83.3%), and saves time by responding to questions
quickly (98%) compared to reference librarians. Fewer than half of the students
agreed or strongly agreed that ChatGPT’s knowledge base is not up to date
(47.2%). Most agreed or strongly agreed that it cannot comprehend some
questions (72.3%), that it cannot read emotions as a librarian would (74.1%),
and that responses to questions may be incorrect (66.6%). The potential
advantage with the strongest response score was that ChatGPT saves time by
responding to questions quickly (mean 3.52). The potential disadvantage with
the strongest response score was ChatGPT could not read emotions as a librarian
would (mean 2.91).
Conclusion – Students
from an academic institution acknowledged the potential advantages and
disadvantages of ChatGPT over reference librarians, yet the majority of
students would continue to utilize reference librarian services. The author
suggests that ChatGPT is a versatile and useful tool as a supplement rather
than a replacement for knowledgeable and personable reference librarians. Based
on the results of the study, the author emphasizes the importance of
interpersonal skills and enhanced accessibility of reference librarians outside
of typical work hours.
Commentary
This research was appraised with Burns and Kho’s
(2015) guide to assessing survey research.
While the author elicited some valuable information
from their survey, the author reported limitations to their study. The sample
was small and from an undergraduate population from one university, thereby
limiting its generalizability. The author stated that they “did not explore
other factors that may influence their [students’] preference for reference
consultations” (Adetayo, 2023, p. 143). This is reflected in the broad nature
of the study questions and limited range of responses available in determining
types of inquiries and perceived advantages and disadvantages of using ChatGPT
rather than the reference librarian. Participants were restricted to five
categories when asked the type of inquiry made while using ChatGPT and
restricted to four options each regarding advantages and disadvantages to using
ChatGPT. These may not have reflected the user’s full perception or use of
ChatGPT. Having a place for open-ended responses would have strengthened the
study, as well as having an option of neither agree nor disagree (5-point
Likert scale). Open-ended responses could have directed the author in future
research on the same topic, with the potential to alter or expand the items
investigated. With regard to measuring frequency of use, defining numerically
the range of frequencies may also have strengthened the study. Terms such as
rarely, occasionally, or frequently may be interpreted differently, leading to
inconsistencies in responses.
This study topic is current and generated findings
significant to the author’s institution. Even though the information elicited
derives from a small sample, the author reports student agreement with benefits
and limitations to ChatGPT that are reflected in other publications (Prathiba,
2021; Yamson, 2023).
The artificial intelligence (AI)-powered chatbot
ChatGPT can be programmed to meet the needs of an institution, has shown value
in answering common questions regarding hours, events, and services, and can
potentially reduce the workload of librarians, leaving them free for more
complex questions (Prathiba, 2021; Yamson, 2023). ChatGPT uses natural
language, is available 24/7, and responds quickly. When asked, it can provide a
list of 10 realistic examples of how it can assist reference librarians in enhancing
the efficiency of library services, including answering reference questions,
language translation, or generating reading lists (OpenAI, 2024). ChatGPT,
however, relies on pre-existing data that was last updated January 2022. When
this reviewer prompted ChatGPT about the limitations of using ChatGPT in the
reference library, the generated response stated that ChatGPT “may not always
provide accurate information, especially for complex or specialized topics,”
that it does not provide the expertise of human librarians, and that it may not
always interpret questions correctly (OpenAI, 2024).
The use of artificial intelligence chatbots, such as
ChatGPT, is increasing. Reference librarians should embrace these tools that
can enhance efficiency and should assist students in using them responsibly.
However, librarians should also be mindful of the limitations of this
technology. Based on the study findings, the author suggests that libraries
prioritize accessibility and investigate the option of providing reference
consultations outside of typical work
hours as well as explore ways to incorporate ChatGPT as a supplement to
traditional reference services. The author advocates for interpersonal
skills training for librarians to build strong communication skills in order to
provide students with the information needed to succeed in a comfortable
atmosphere.
References
Adetayo, A. J. (2023). ChatGPT and librarians for reference
consultations. Internet Reference Services Quarterly, 27(3),
131–147. https://doi.org/10.1080/10875301.2023.2203681
Burns, K. E. A., & Kho, M. E. (2015). How to assess a survey report:
A guide for readers and peer reviewers. CMAJ: Canadian Medical
Association Journal, 187(6), E198–E205. https://doi.org/10.1503/cmaj.140545
OpenAI. (2024). ChatGPT (Version 3.5) [Large
language model].https://chat.openai.com
Prathibha, S. N., & Shilpa Rani, N. R. (2021). ChatGPT: A boon to library services. LIS Links Newsletter, 7(1),
8–13. http://file.lislinks.com/newsletter/lislinks-newsletter-vol-7-no-1-p-8-13.pdf
Yamson, G. C. (2023). Immediacy as a better service: Analysis of
limitations of the use of ChatGPT in library services. Information
Development. Advance online publication. https://doi.org/10.1177/02666669231206762