Academic Libraries’ Citation Guides to ChatGPT Show Mixed Levels of Accuracy and Currency

Abstract

A Review of:

Moulaison-Sandy, H. (2023). What is a person? Emerging interpretations of AI authorship and attribution. Proceedings of the Association for Information Science & Technology, 60(1), 279–290. https://doi.org/10.1002/pra2.788

Objective – To examine how and which academic libraries are responding to emerging guidelines on citing ChatGPT in the American Psychological Association (APA) style through guidance published on the libraries’ websites. 

Design – Analysis of search results and webpage content.

Setting – Websites of academic libraries in the United States.

Subjects – Library webpages addressing how ChatGPT should be cited in APA format.

Methods – Google search results for academic library webpages providing guidance on citing ChatGPT in APA format were retrieved on a weekly basis using the query “chatgpt apa citation site:.edu” over a six-week period that covered the weeks before and immediately after the APA issued official guidance for citing ChatGPT. The first three pages of relevant search results were coded in MAXQDA and analyzed to determine the type of institution, using the Carnegie Classification and membership in the Association of American Universities (AAU). As this was a period during which APA style recommendations for citing ChatGPT were shifting, the accuracy of the library webpage content was also assessed and tracked across the studied time period.

Main Results – During the six-week period, the number of library webpages with guidance for citing ChatGPT in APA format increased. Although doctoral universities accounted for the largest number of webpages each week, baccalaureate colleges, baccalaureate/associate’s colleges, and associates’ colleges were also well-represented in the search results. Institutions belonging to the AAU were represented by a relatively small number throughout the study. Over half of the pages made some mention of APA’s recommendations being interim or evolving, though the exact number fluctuated throughout the period. Prior to the collection period, APA had revised its initial recommendations to cite ChatGPT as a webpage or as personal communication, but 40% to 60% of library webpages continued to offer this outdated guidance. Of the library webpages, 13% to 40% provided verbatim guidance from ChatGPT responses on how it should be cited. The final two weeks of the collection period occurred after April 7, 2023, when APA had published official recommendations for citing ChatGPT. In the week following this change, none of the webpages in the first three pages of results had been updated to fully capture the new recommendations. The study analyzed the nine webpages appearing in the first page of results for the second week after APA’s official recommendations were published, showing that three linked to the APA’s blog, zero provided further explanation on how to apply the recommendations, five included outdated guidance, and three gave guidance from ChatGPT’s responses to questions on how it should be cited.

Conclusion – The author sees the results of the study as reflecting three interrelated components: a new technology, gaps in librarians’ knowledge related to large language models (LLMs) and how they are currently being discussed in terms of authorship, and Google’s inability to rank the results in a way that prioritizes correct information. The substantial presence of institutions serving undergraduates leads the author to conclude that this is the population most in need of guidance for citing ChatGPT and the responsiveness on the part of the librarians shows an understanding of this need, even if the guidance itself is inaccurate.