Evidence Summary

 

ChatGPT not Useful as a Tool to Streamline Library Cataloguing Processes

 

A Review of:

Brzustowicz, R. (2023). From ChatGPT to CatGPT: The Implications of Artificial Intelligence on Library Cataloging. Information Technology and Libraries, 42(3). https://doi.org/10.5860/ital.v42i3.16295

 

Reviewed by:

Andrea Miller-Nesbitt

Associate Librarian

Schulich Library of Physical Sciences, Life Sciences, and Engineering

McGill University

Montreal, Quebec, Canada

Email: andrea.miller-nesbitt@mcgill.ca

 

Received: 3 Mar. 2024                                                               Accepted:  16 May 2024

 

 

Creative Commons logo 2024 Miller-Nesbitt. This is an Open Access article distributed under the terms of the Creative Commons‐Attribution‐Noncommercial‐Share Alike License 4.0 International (http://creativecommons.org/licenses/by-nc-sa/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one.

 

 

DOI: 10.18438/eblip30524

 

 

Abstract

 

Objective – To evaluate the potential of ChatGPT as a tool for improving efficiency and accuracy in cataloguing library records.

 

Design – Observational, descriptive study.

 

Setting – Online, using ChatGPT and the WorldCat catalogue.

 

Subject – The Large Language Model (LLM) ChatGPT.

 

Methods – Prompting ChatGPT to create MARC records for items in different formats and languages and comparing the ChatGPT derived records versus those obtained from the WorldCat catalogue.

 

Main results – ChatGPT was able to generate MARC records, but the accuracy of the records was questionable, despite the authors’ claims.

 

Conclusion – Based on the results of this study, the author concludes that using ChatGPT to streamline the process of cataloging could allow library staff to focus time and energy on other types of work. However, the results presented suggest that ChatGPT introduces significant errors in the MARC records created, thereby requiring additional time for cataloguers to correct the error-laden records. The author correctly stresses that if ChatGPT were used to assist with cataloguing, it would remain important for professionals to check the records for completion and accuracy.

 

Commentary

 

This article resulted in three separate letters to the editor criticizing the methods and conclusions (DeZelar-Tiedman, 2023; Amram et al., 2023; Floyd, 2023). All three of these responses assert that ChatGPT would not be a useful tool for streamlining the cataloguing process, as the MARC records generated by the LLM are riddled with errors. These letters to the editor go into technical detail about the shortcomings of the study from the perspectives of professional cataloguers.

 

The literature review of the study is very limited, with only four articles cited in this section. The articles cited discuss ChatGPT broadly, and uses of ChatGPT more specifically in the context of higher education and libraries. Given that ChatGPT is a relatively new technology, and therefore not a great deal of literature exists on the topic yet, it would have been interesting for the author to zoom out and situate the current study in the historical context. For example, Weible (1990) and Burger (1984) discuss automation of cataloguing. More recent articles such as Lowagie (2023) are also missing from the literature review, however it is possible they were published after this study was submitted for review.

 

This study was appraised using the CAT critical appraisal tool by Perryman and Rathbun-Grubbs (2014). The research objective is clear, and the methodology is appropriate, if limited (as discussed below). The first item selected by the author to test ChatGPT as a cataloguing tool is one that is relatively simple (i.e. readily available, English language). Subsequent items used become increasingly complex (i.e. different formats, non-English language items, non-Latin characters), with the final item chosen being one with no existing WorldCat entry.  

 

This study shows that ChatGPT can create catalogue records albeit records with significant errors. The author maintains the importance of reviewing and editing MARC records created by ChatGPT to ensure that they are complete (e.g. replacing placeholder content) and correct any biases or inaccuracies. In their letters to the editors, DeZelar-Tiedman (2023), Amram et al., (2023), and Floyd, (2023) maintain that the time required to correct the inaccuracies in the ChatGPT-created records would negate any perceived efficiencies.

 

Various limitations in ChatGPT itself are discussed. For example, the risk of bias based on the data used to train the LLM, questions around copyright infringement and crediting intellectual labour creating catalogue records, issues of privacy, and ownership of generated content are identified as issues that need to be carefully considered before ethically using ChatGPT for cataloguing. The author does not mention any limitations in the study itself. Regarding the methods section, the sample size is too small to be generalizable and the author does not provide the prompts used to query ChatGPT. Regarding the conclusions, the fact that the ChatGPT generated records differ from the WorldCat records, in some instances quite significantly, is not addressed.

 

This study, despite its shortcomings, does contribute to a conversation about the use of Artificial Intelligence and Large Language Models in cataloguing. Doing a similar study with a larger sample size, more transparent methodology, and more critical interpretation of results would be interesting.

 

References

 

Amram, T., Malamud, R. G., & Hollingsworth, C. (2023). Response to "From ChatGPT to CatGPT". Information Technology and Libraries, 42(4). https://doi.org/10.5860/ital.v42i4.16983

 

Brzustowicz, R. (2023). From ChatGPT to CatGPT: The Implications of Artificial Intelligence on Library Cataloging. Information Technology and Libraries, 42(3). https://doi.org/10.5860/ital.v42i3.16295

 

Burger, R. H. (1984). Artificial Intelligence and Authority Control. Library Resources and Technical Services, 28(4), 337-45.

 

DeZelar-Tiedman, C. (2023). Response to "From ChatGPT to CatGPT". Information Technology and Libraries, 42(4). https://doi.org/10.5860/ital.v42i4.16991

 

Floyd, D. (2023). Response to "From ChatGPT to CatGPT". Information Technology and Libraries, 42(4). https://doi.org/10.5860/ital.v42i4.16995

 

Lowagie, H. (2023). From Bias to Transparency: Ethical Imperatives in AI-Based Library Cataloging. Retrieved from https://repository.ifla.org/handle/123456789/2841

 

Perryman, C. & Rathbun-Grubb, S. (2014). The CAT: a generic critical appraisal tool. In JotForm – Form Builder. Retrieved 21 Aug. 2014 from http://www.jotform.us/cp1757/TheCat

 

Weibel, S. L. (1990). Automated cataloging: implications for libraries and patrons. 27th Clinic on Library Applications of Data Processing, (pp. 67-80). Retrieved from http://hdl.handle.net/2142/1294