Abstract

Objective – To examine how and which academic libraries are responding to emerging guidelines on citing ChatGPT in the American Psychological Association (APA) style through guidance published on the libraries’ websites.

Design – Analysis of search results and webpage content.

Setting – Websites of academic libraries in the United States.

Subjects – Library webpages addressing how ChatGPT should be cited in APA format.

Methods – Google search results for academic library webpages providing guidance on citing ChatGPT in APA format were retrieved on a weekly basis using the query “chatgpt apa citation site:.edu” over a six-week period that covered the weeks before and immediately after the APA issued official guidance for citing ChatGPT. The first three pages of relevant search results were coded in MAXQDA and analyzed to determine the type of institution, using the Carnegie Classification and membership in the Association of American Universities (AAU). As this was a period during which APA style recommendations for citing ChatGPT were shifting, the accuracy of the library webpage content was also assessed and tracked across the studied time period.

Main Results – During the six-week period, the number of library webpages with guidance for citing ChatGPT in APA format increased. Although doctoral universities accounted for the largest number of webpages each week, baccalaureate colleges, baccalaureate/associate’s colleges, and associates’ colleges were also well-represented in the search results. Institutions belonging to the AAU were represented by a relatively small number throughout the study. Over half of the pages made some mention of APA’s recommendations being interim or evolving, though the exact number fluctuated throughout the period. Prior to the collection period, APA had revised its initial recommendations to cite ChatGPT as a webpage or as personal communication, but 40% to 60% of library webpages continued to offer this outdated guidance. Of the library webpages, 13% to 40% provided verbatim guidance from ChatGPT responses on how it should be cited. The final two weeks of the collection period occurred after April 7, 2023, when APA had published official recommendations for citing ChatGPT. In the week following this change, none of the webpages in the first three pages of results had been updated to fully capture the new recommendations. The study analyzed the nine webpages appearing in the first page of results for the second week after APA’s official recommendations were published, showing that three linked to the APA’s blog, zero provided further explanation on how to apply the recommendations, five included outdated guidance, and three gave guidance from ChatGPT’s responses to questions on how it should be cited.

Conclusion – The author sees the results of the study as reflecting three interrelated components: a new technology, gaps in librarians’ knowledge related to large language models (LLMs) and how they are currently being discussed in terms of authorship, and Google’s inability to rank the results in a way that prioritizes correct information. The substantial presence of institutions serving undergraduates leads the author to conclude that this is the population most in need of guidance for citing ChatGPT and the responsiveness on the part of the librarians shows an understanding of this need, even if the guidance itself is inaccurate.

Commentary

As artificial intelligence (AI) has become increasingly available and more widely used in recent years, librarians have responded by considering its application to library operations as well as its relevance to user needs. Lo (2023) included “provid[ing] AI literacy education and training” among a set of practical recommendations for libraries as they adapt to increasing AI usage. Andersdotter (2023) explained the growing need for including AI in information literacy instruction, noting “libraries need to match this societal development with relevant knowledge dissemination.” The current study connects this identified role for librarians to educate users about AI with an information literacy practice common in academic libraries, providing guidance for correctly citing sources in specific styles (Moulaison-Sandy, 2023).

The CAT: A Generic Critical Appraisal Tool is used in this evidence summary to assess the quality of the study, finding it to be high (Perryman & Rathbun-Grubb, 2014). Moulaison-Sandy has published previously on the topic of authorship, an area of discussion that is evolving as organizations with influence over ideas of authorship consider the nature of large language models (LLMs) like ChatGPT and implications for related concepts such as attribution, originality, and copyright. This expertise is evident in the authors road examination of how authorship is established and regarded throughout multiple realms including scholarly publishing, government policy, media, and education.

As examined in the Generic CAT’s section on background information, a literature review is used not only to provide an overview of existing sources related to LLMs and authorship but also to provide an impetus for inspecting libraries’ responses to evolving guidance for citing ChatGPT in American Psychological Association (APA) format. Moulaison-Sandy notes that the question of authorship and LLMs is new enough to not have generated a substantial amount of peer-reviewed research and appropriately fills this gap with credible and relevant sources that include the U.S. Copyright Office, editorials from highly regarded academic journals, and style guides. The author correctly sees style guide interpretation of LLM authorship as fulfilling a practical requirement for ChatGPT end users, as they have a need to cite the tool in their writing. As guidance on citing information in various styles, especially APA format, is often provided through academic library web pages, Moulaison-Sandy’s inquiry into how and by whom this guidance is presented provides relevant insights for understanding librarians’ responses, as well as potential impacts on students.

By using Google to retrieve search results, the author demonstrates aspects of appropriate data collection examined by the Generic CAT by following a likely pathway for students looking for information on citing ChatGPT. Limiting data collection to the first three pages of results also helps to ensure a pool of results from which it is reasonable to assume that students will select the guidance that they ultimately use. Although the author’s choice to not allow Google to personalize results is not one likely to be seen in real-world searches conducted by students, this is an important consideration for reducing biased results. The time period during which the data was collected, the weeks preceding and following APA’s publication of official guidelines for citing ChatGPT, is also an especially useful choice for gaining insight to librarians’ responsiveness to this guidance.

Among the results, two facets stand out as having especially significant implications for practice. First, the surprising number of results from institutions serving only undergraduate students signals the user demographic for whom this kind of guidance might be most strongly needed. Even institutions granting doctoral, and master’s degrees can benefit from identifying and addressing issues that may have special relevance for their undergraduate population. Second, while webpages displaying outdated information were concerning, webpages with incorrect information (asking ChatGPT itself how it should be cited) indicate a need for greater training and education among librarians, echoed elsewhere, especially as use of ChatGPT and other LLMs continues to proliferate in higher education. Many libraries choose not to present original content on using or interpreting citation style guidelines, instead opting to link to web pages at other institutions. Findings related to outdated or incorrect content should prompt libraries to ensure that however guidance for citing ChatGPT is presented, it is both updated and accurate.

References

Andersdotter, K. (2023). Artificial intelligence skills and knowledge in libraries: Experiences and critical impressions from a learning circle. Journal of Information Literacy, 17(2), Article 2. https://doi.org/10.11645/17.2.14

Lo, L. S. (2023). AI policies across the globe: Implications and recommendations for libraries. IFLA Journal, 49(4), 645–649. https://doi.org/10.1177/03400352231196172

Moulaison-Sandy, H. (2023). What is a person? Emerging interpretations of AI authorship and attribution. Proceedings of the Association for Information Science and Technology, 60(1), 279–290. https://doi.org/10.1002/pra2.788

Perryman, C., & Rathbun-Grubb, S. (2014). The CAT: A generic critical appraisal tool. https://form.jotform.us/42065968239162

Academic Libraries’ Citation Guides to ChatGPT Show Mixed Levels of Accuracy and Currency

Abstract

Commentary

References