This article has been corrected. View the correction note.

An Exploration of Journals Requested by Health Sciences Libraries through DOCLINE Interlibrary Loan During the Early COVID-19 Pandemic

Caitlin J. Bakker
Discovery Technologies Librarian
Dr. John Archer Library
University of Regina
caitlin.bakker@uregina.ca

Jessica A. Koos
Health Sciences Librarian
Interim Head of Health Sciences Library
Health Sciences Library
Stony Brook University
jessica.koos@stonybrook.edu

Margaret A. Hoogland
Clinical Medical Librarian
University Libraries
University of Toledo
margaret.hoogland@utoledo.edu

Debra Rand
Associate Dean for Library Services
Zucker School of Medicine at Hofstra/Northwell
drand@northwell.edu

Kristine M. Alpi
Associate Dean for Libraries & Information Sciences
Icahn School of Medicine at Mount Sinai
kris.alpi@mssm.edu

Correction

Data values in Tables 1 and 4 were edited on January 9, 2024 at the request of the authors to correct publishing errors.

Abstract

Keywords: Interlibrary loan, Health sciences libraries, DOCLINE, Technical services

Recommended citation:

Bakker, C. J., Koos, J. A, Hoogland, M. A., Rand, D., & Alpi, K. M. (2023). An exploration of journals requested by health sciences libraries through DOCLINE interlibrary loan during the early COVID-19 pandemic. Issues in Science and Technology Librarianship, 103. https://doi.org/10.29173/istl2744

Introduction

Health sciences libraries (HSLs) in North America responded quickly in March 2020 when the COVID-19 pandemic closed many academic institutions and hospital libraries. While some libraries remained open with limited staffing, most library staff pivoted to remote work and set up procedures to maintain services including interlibrary loan (ILL) and document delivery (DD) (Lloyd et al., 2022). On health sciences library discussion lists, questions surfaced regarding potential effects on ILL/DD activity, particularly fill rates, characteristics of unfilled requests such as year of publication or subject area, and cost factors (Creazzo et al., 2020).

A tripartite research agenda was developed to determine how COVID-19 impacted ILL/DD activity for HSLs, including the impact on internal workflows and policies, and the impact on users and their ability to access information. Three groups undertook this work: one administering a survey to understand librarian attitudes and practices (Lloyd et al., 2022); one examining borrowing data from DOCLINE; and one examining local ILL/DD data not captured through DOCLINE. The intention of this tripartite approach was to examine the impact of COVID-19 on ILL/DD from a number of different angles, including librarian perceptions and reported practices, centralized data sources, and locally collected data. By approaching the question from multiple angles and with multiple research methods, we aim to more fully address the overall question. This article focuses on the work of the Secondary Data Analysis (SDA) team, which examined DOCLINE borrowing data.

The SDA team performed an exploratory analysis of aggregated ILL borrowing data from 15 HSLs during the pandemic to determine if the proportion of filled and unfilled requests changed between 2019 and 2020. Additionally, the SDA team sought to determine whether the nature of the requests (age of the articles, language of the articles, most requested journals, and subject areas of the journals) changed.

Based on the closure of many large research libraries, the SDA team hypothesized that requests for older materials and materials in languages other than English were less likely to be filled in 2020 when compared to 2019. For unfilled requests, the SDA team collaborated with EBSCO to determine availability of backfiles, cost of purchasing or gaining short-term access, and overall costs associated with filling requests through electronic subscriptions.

Literature Review

No library can provide immediate access to all resources that its patrons may want or need. Decreasing or flat budgets, combined with the increasing cost and variety of resources, place strain on libraries in fulfilling their core mission of connecting patrons and information. The Association of Research Libraries found that between 1997 and 2017, total library expenditures as a percentage of total university expenditures decreased approximately 44%, from 3% to 1.7%, while total library materials expenditures increased by 123% between 1998 and 2018 (Monroe, n.d.). In an environment of continually limited budgets, increasing costs and diversifying needs, resource sharing and collaborative collection management are essential mechanisms by which libraries can meet patron needs.

Over 2,000 libraries in Canada and the United States participate in DOCLINE, which is the National Library of Medicine’s (NLM) ILL routing system (National Library of Medicine, 2017). Since its inception in 1985, libraries have established profiles in DOCLINE and report their journal holdings in the system. These libraries are subsequently able to submit requests, which are then matched with participating libraries who may be able to fill the request using an algorithm that matches requests to the reported journal holdings. NLM, and its associated National Network of Libraries of Medicine, have been referred to as “a kind of national consortium” for health sciences libraries, and DOCLINE has “strengthened the links between libraries by providing a quicker, automated method of routing interlibrary loan (ILL) requests.... [DOCLINE] provided the infrastructure to support the effective resource sharing that is essential to the success of any collaboration” (Carrigan et al., 2018, p. 149-150). This collaborative infrastructure allowed libraries to evaluate and develop their collections in the context of the consortia, considering overlap and duplication with other participating libraries when making selection and retention decisions.

While librarians had grappled with issues of adapting tools and practices to an electronic environment (Munson, 2012), sharing with colleagues in international settings (Sharing and Transforming Access to Resources Section International Interlibrary Loan Committee, 2009), and the need to educate users on ILL services (O’Brien, 2004), the COVID-19 pandemic introduced a previously unseen challenge: widespread library closures and inaccessible print collections. For the first time since the introduction of these automated methods and shared infrastructure, many print collections were either totally unavailable or were accessible on a delayed basis as staff returned on a limited schedule to on-site libraries (Lloyd et al., 2022).

In the early stages of the pandemic, many publishers provided free online access to their content (Huffman, 2020). While free electronic resources could aid libraries in meeting patron needs virtually, this access was only temporary and created concern about how those needs could be met once free access was no longer an option (Fredericksen, 2020). Furthermore, although electronic collections may seem more appealing due to ease of access, print collections should not be totally disregarded in light of the possibility that future disasters may affect access to electricity, as well as the Internet (Hall, 2020).

Although NLM estimates that only 0.82% of requests received in 2020 were associated with COVID-19 (Hochstein et al., 2020), the ILL system was noticeably strained. Harnegie (2021) describes how many medical libraries initially ceased ILL operations involving physical collections completely during the COVID-19 pandemic, and only filled requests remotely. Many institutions then began coordinating library staff schedules so that requests from the physical collection could be filled by one person being onsite at a time to meet user needs, and allow staff to remain employed (Lloyd et al., 2022). Institutions that did not suspend ILL services during the pandemic reported receiving an abundance of requests from other libraries, as so many had closed or offered limited services, and an increased number of unfilled requests (Howes et al., 2021). Conversely, one institution that restricted ILL and imposed a work-from-home mandate reported that article borrowing increased by 37% compared to the same time the previous year while article lending decreased significantly (Koos et al., 2021). One survey of 875 libraries found that, while access to print collections has been largely restored, this access was staff-mediated in approximately 40% of cases (Lutz, 2020). Such workflow changes require increased staff resources to enable access to the library collections.

Throughout the pandemic, libraries have faced financial challenges and the need to maintain or increase access to educational content for faculty, staff, and students (Allison et al., 2020). In a contemporary survey about ILL practices during the early COVID pandemic, 23% of respondents purchased items directly from publishers and 15% purchased items through an existing arrangement with a third-party provider (Lloyd et al., 2022). While these purchases may provide an immediate solution, the availability of materials for purchase and the scalability and sustainability of these models has yet to be determined.

A survey of library directors revealed that not only were budgets cut in the short-term, but the effects of such cuts may be long-term and could limit the acquisition of electronic access to articles (Frederick & Wolff-Eisenberg, 2020). A range of potential collection development approaches have been proposed in response to COVID-19 and continuing budget challenges. While some information professionals advocate for a collection development strategy that encompasses only purchasing essential resources (Schonfield, 2020), others stress the importance of libraries in supporting open access business models, as traditional online collections will become increasingly cost prohibitive, particularly when factored in with post-COVID budget constraints (Ohler & Pitts, 2021).

Historically, when faced with financial challenges that would necessitate print or electronic journal cancellations, libraries relied on ILL to provide ongoing access to canceled titles (Knowlton et al., 2015; Nabe & Fowler, 2015). When considering budget constraints in the context of a shifting ILL landscape, it is necessary to understand what gaps in ILL coverage emerged in the early pandemic.

Research Objectives

To understand these potential gaps in ILL coverage more fully, we sought to answer the following questions. In this sample of DOCLINE participating institutions:

1. Were there a significantly higher number of unfilled requests in 2020 compared to 2019?
2. Did fill rates differ across subject areas between 2020 and 2019?
3. Was the proportion of unfilled requests for older journals significantly higher in 2020 than in 2019?
4. Was the proportion of unfilled requests in languages other than English significantly higher in 2020 than in 2019?
5. Were unfilled requests available for print or online purchase and, if so, for what price?
6. What was the availability of backfiles for unfilled requests?

Methods

The SDA team conducted a retrospective data analysis of aggregate HSL ILL borrowing requests for the comparison periods of March – August 2019 and March – August 2020. The period of March – August for each year was chosen to reflect the move to the new NLM DOCLINE system. After a thorough vetting process to ensure patron privacy was maintained, NLM agreed to share aggregated ILL data for a maximum of 15 libraries. The SDA team publicized the study opportunity widely and 12 academic libraries, 2 hospital libraries, and 1 association library agreed to participate and share ILL data. Of those libraries, four primarily used DOCLINE. The other 11 libraries used a combination of DOCLINE and OCLC products for interlibrary loan. Oregon Health & Science University’s Institutional Review Board determined the study did not meet the criteria for human subjects research.

The study data consisted of aggregate borrowing data from the 15 participating institutions. This included the number of filled and unfilled requests with fields for journal title, International Standard Serial Number (ISSN), NLM Unique IDs, and publication year. No identifiable information regarding the institution was made available. NLM also provided system-wide fill rates. The data was cleaned and normalized by three researchers, using OpenRefine and Excel.

The SDA team obtained subscription cost information for the journal titles and explored backfile availability by matching ISSNs and coordinating with EBSCO Information Services. To determine differences between requests for articles in English compared with other languages, the SDA team obtained journal-level language data from the NLM Catalog. For source titles, which were non-journals or variants of publications in the NLM Journal database, the team searched in OCLC Worldcat or the publication website by one of the authors to identify the language of publication for such titles. Three citations were not assigned a language, as they included the name of a company rather than a journal.

We dichotomized journal age as newer (less than 5 years old) and older (greater than 5 years old). This threshold was based on previous research which found that the majority of usage of journals in the health sciences occurs within the first 5 years following their publication (De Groote, 2008; Tsay, 1999). The SDA team used this data to calculate descriptive statistics and odds ratios (OR) and conduct 2-proportion z-tests to determine differences in the data before and during the first year of the COVID-19 pandemic.

Results and Discussion

NLM provided system-wide request volume and outcome as context for the results of our 15-library study population shown in Table 1. Total requests decreased by a similar proportion between 2019 and 2020 both system-wide (16.7%) and in the study population (17.8%).

Requests	Study Population		DOCLINE System-Wide
	2019	2020	2019	2020
Unfilled Requests	380	779	29,803	35,694
Filled Requests	7,795	5,937	400,529	322,628
Total Requests	8,175	6,716	430,332	358,322

Both system-wide and across our sample, ILL requests decreased significantly from 2019 to 2020. This could be attributed to disruption associated with the onset of the pandemic, including rapid shifts to remote learning and working, the need to balance work with increased personal responsibilities, or a change in clinical, research or administrative duties specific to the pandemic (Bahadirli & Sagaltici, 2021; Foley & Tippett, 2020; Thompson et al., 2021). DOCLINE requests have been decreasing annually since 2002, although more modestly, with decreases such as 7.6% between 2015 and 2016 and 6.0% between 2016 and 2017 (Collection Access Section, National Library of Medicine, 2018).

Proportions of Filled and Unfilled Requests

As hypothesized, unfilled requests in the early pandemic rose system-wide from 6.9% to 10.0% and in our study population from 4.6% to 11.6%. System-wide, the odds of having an unfilled request were 1.5 times higher (OR=1.49, 95% CI=1.46-1.51, p < 0.001) in 2020 than 2019. In the study population, the odds of an unfilled request were 2.7 times higher in 2020 (OR=2.69, 95% CI=2.37-3.06, p < 0.001). These findings may be indicative of changes in ILL filling behaviors by libraries. One survey found that 45% of libraries ceased filling interlibrary loan requests for print materials during the pandemic (Lloyd et al., 2022). Once libraries transitioned to remote work, many continued to fill online-only requests (Schonfield, 2020). Such changes in library practices may result in higher rates of unfilled requests.

Subject Area of the Journals

Considering the study population data, in both 2019 and 2020, journals indexed with the Medical Subject Heading (MeSH) Medicine were the most common (2019 = 452/8175, 5.5%, 2020 = 470/6716, 7.0%). Dentistry accounted for 2.4% (196/8175) of requests in 2019 and 3.7% (246/6716) of requests in 2020, while Nursing accounted for 1.2% (96/8175) in 2019 and 1.1% (72/6716) in 2020. Fill rates for journals with these subject headings paralleled overall fill rates. Fill rates for Medicine journals decreased from 95.6% in 2019 (432/452) to 82.3% in 2020 (387/470), while fill rates for Dentistry decreased from 94.9% (186/196) to 83.3% (205/246) and Nursing fill rates decreased from 89.6% (86/96) to 84.7% (61/72).

Subject Areas	2019		2020
	Number	%	Number	%
Health sciences only	3,165	39	2,487	37
Health sciences and one other discipline	1,585	19	1,060	16
Non-health sciences journals	630	8	485	7
Unclassified	2,795	34	2,684	40
Total	8,175	100	6,716	100

As seen in Table 2, when considering journals classified using the multidisciplinary All Science Journal Classification (ASJC), the majority of journals requested in both 2019 and 2020 were for health-sciences only journals. Health sciences journals in 2019 had a fill rate of 96.4% (169 unfilled of 4,750 requests), while non-health sciences journals had a fill rate of 95.6% (28 of 630). In 2020, fill rates declined to 92.7% for health sciences journals (260 of 3,547), 94.4% for non-health sciences journals (27 of 485).

Age of the Articles

When considering the proportion of requested articles that are more than five years old as seen in Table 3, there are statistically significant differences between filled, unfilled, and total requests in 2019 and 2020. There are higher proportions of older articles requested in 2020 than 2019 (overall = 47.8% versus 55.2%, p < .001; filled = 48.0% versus 53.4%, p < .001; unfilled = 44.2% versus 69.3%, p < .001). While there was no statistically significant difference in the proportion of older articles filled or unfilled in 2019 (Z = 1.45, p = .15), the difference in 2020 between the proportion of older articles in the filled and unfilled requests (Z = -8.41, p < .001) was statistically significant.

Requests	2019 Items Published 2014 or Earlier		2020 Items Published 2015 or Earlier		Comparison of 2019 and 2020
	Number	%	Number	%
Unfilled Requests	168 / 380	44.2	540 / 779	69.3	Z = -8.23 (p < .001)
Filled Requests	3743 / 7795	48.0	3,169 / 5,937	53.4	Z = -6.22 (p < .001)
Total Requests	3,911 / 8,175	47.8	3,709 / 6,716	55.2	Z = -8.97 (p < .001)

The rate of unfilled requests for older items rose significantly (Z = -15.4252, p < .001) from 4.3% (168/3911) in 2019 to 14.6% (540/3709) in 2020. This increased rate of unfilled requests for older articles may reflect the fact that more recent titles are often available electronically, while libraries may rely on print for older titles which may have not been available with closures of physical spaces (Harnegie, 2021).

Language of the Articles

The proportion of total requests for material in languages other than English between 2019 and 2020 (Table 4) remained stable at approximately 12%. For unfilled requests overall, 16.3% of requests in 2019 were for these languages, compared to 32.7% in 2020 (p < .001). As hypothesized, articles in languages other than English were less likely to be filled in 2020. The 2.4% decrease in proportion of filled requests, 11.7% in 2019 compared to 9.3% in 2020 was significantly lower (p < .001). The rate of unfilled requests for items that were in languages other than English rose significantly (Z = -13.84267, p < .001) from 6.4% (62/971) in 2019 to 31.6% (255/806) in 2020.

Requests	2019		2020		Comparison of 2019 and 2020
	Number	%	Number	%
Unfilled Requests	62 / 380	16.3	255 / 779	32.7	Z = -5.89 (p < .001)
Filled Requests	909 / 7,795	11.7	551 / 5,937	9.3	Z = 4.48 (p < .001)
Total Requests	971 / 8,175	11.9	806 / 6,716	12.0	Z = 0.2314 (p = .818)

There are several possible reasons for libraries in the United States to prioritize English language content, including limited space and financial resources, or language skills of their primary audiences. Robust non-English language collections are often housed in large academic libraries, many of which were closed during the pandemic. Libraries without these collections who had relied on their ability to borrow to supplement their collections found that this solution was no longer available to them. The lack of access and subsequent exclusion of content in other languages from systematic reviews may lead to omitting evidence, an increased risk of bias, and limited generalizability (Walpole, 2019).

Most Requested Journals

In 2019, requests were placed for content from 3,037 unique journals compared with 2,873 journals in 2020. In both periods, just over half the journal titles were requested once (2019, n=1,551, 51.1%; 2020, n=1,588, 55.3%). In 2019, each journal was requested a mean of 2.65 times, while in 2020 the mean was 2.33 times. Focusing specifically on the journal titles that were unfilled in 2020, there were 779 unfilled requests from 544 unique titles, compared to 380 unfilled requests from 279 unique titles in 2019. In both 2019 and 2020, each unfilled journal title was requested a mean of 1.4 times. The majority of unfilled journals in both periods were requested only once (2019, n=217, 77.8%; 2020, n=416, 76.5%).

Price of Subscriptions for Journals with Unfilled Requests

When considering print plus online subscription prices, there is no statistically significant difference between 2019 and 2020 ($2,075 ± $2,423 versus $2,038 ± $2,013, p = .81). Looking at online-only prices for unfilled journal titles, as expected with annual inflation, the average subscription price in 2020 was significantly higher than in 2019 ($1,815 ± $2,167 versus $2,132 ± $2,741, p = .047), a 17.5% difference in average price. Online-only subscription prices ranged from $115 to $11,090 in 2019, and $95 to $18,687 in 2020. Print plus online subscription prices ranged from $112 to $12,200 in 2019, and $47 to $13,956 in 2020.

Availability of Digital Backfiles

Of the 380 journals with unfilled requests in 2019 and the 779 in 2020 (Table 5), information on backfile availability was available for 202 and 317 titles, respectively. In 2020, the odds of a backfile for an unfilled request being unavailable was 1.5 times higher than in 2019 (95% CI: 1.02 - 2.11, p = .04). Though there was an increase in the proportion of unavailable backfiles in 2020 compared to 2019, this was not statistically significant.

Backfile Availability*	Unfilled Journals in 2019		Unfilled Journals in 2020		Comparison of 2019 and 2020
	Number	%	Number	%
Backfiles available	129 / 202	63.9	173 / 317	54.6	Z = 1.206 (p = 0.523)
Backfiles not available	73 / 202	36.1	144 / 317	45.4	Z = -1.3084 (p = 0.951)
*Source: 2021 backfile availability data provided by EBSCO

Acquiring subscriptions or purchasing backfiles to satisfy unfilled requests would be an expensive proposition for an individual library. Across our sample of 15 libraries, 51.1% of journals in 2019 were requested once, which rose to 55.3% in 2020. A single request would not warrant consideration of a subscription. More frequently requested titles may prove more viable candidates for acquisition. In 2019, the Journal of Public Child Welfare had the greatest number of unfilled requests (5), and its subscription price is $486 for online-only access ($97 per request) or $572 for print plus online access ($114 per request). Journal of Biological Regulators and Homeostatic Agents had the highest number of unfilled requests in 2020 (10), but its subscription cost was not available. Kardiologiia was the journal with the highest number of unfilled requests (6) in 2020 for which a subscription cost was available ($123 for online-only or $21 per request, print plus online pricing not available). While previous research has established an average pay-per-view (PPV) cost of $37.72 (Lemley & Li, 2015), individual libraries and consortia may have alternative thresholds for determining the point at which purchasing a subscription becomes the more cost-effective option. It is unclear whether the unfilled titles requested in our study would have been available through a third-party provider such as Research Solutions/Reprints Desk (Research Solutions, n.d.).

Limitations

Our sample is not representative of all health sciences libraries participating in DOCLINE. In 2017, DOCLINE-participating hospital libraries predominated with 57% compared to 27% academic and 15% other (Collection Access Section, National Library of Medicine, 2018). The majority of libraries in our sample were academic, with a high proportion of doctoral universities with very high research activity. As such, our results may not be generalizable to non-academic or less research-intensive institutions. As we were accessing aggregate data, it was not possible to determine differences in ILL behaviors between different types of institutions. We are currently investigating the data of the subset of 11 academic libraries who used a system that permits a more granular analysis of ILL data and behaviors.

Institutions conduct ILL services differently. Some have separate interlibrary loan and document delivery services, and others integrate services with their parent academic library. These different ILL models could have influenced our results. The SDA team also considered but ultimately decided against determining the costs for each of the participating libraries for this reason (Allison et al., 2020).

Conclusion

Our study found a statistically significant increase in the proportion of unfilled ILL requests between 2019 and 2020, with the odds of an unfilled request being 2.7 times higher in 2020 than in 2019. The majority of requests were for journals focused on the health sciences, and fill rates for journals in all subject areas declined between 2019 and 2020. Older materials and materials published in languages other than English accounted for a larger proportion of unfilled requests in 2020 when compared to 2019. The higher levels of unfilled requests may be the result of libraries having limited ILL operations and inaccessible physical collections. For older materials, which are more frequently print-based, and materials in languages other than English, which may be less frequently collected in North American libraries, these limited ILL operations have a more pronounced impact.

Acquiring subscriptions and purchasing backfiles are potential solutions when addressing high rates of unfilled ILL requests. In our study, although the cost of print plus online subscriptions for unfilled requests did not increase significantly between 2019 and 2020, the cost of online-only subscriptions did significantly increase. The feasibility of acquiring a subscription depends upon many factors unique to an institution and its context. Collaborative collection development structures for print retention and access were not always able to be leveraged with the staffing constraints of the global pandemic. While the high cost of these resources in our small sample would likely make subscriptions an ineffective use of financial resources to meet local needs, extrapolating from 15 libraries to the approximately 2000 in DOCLINE (Collection Access Section, National Library of Medicine, 2018) suggests that the number of requests might warrant a subscription or backfile purchase by a consortium or other regional network serving the needs of many libraries.

While libraries converting from primarily print or print-and-electronic journals have previously considered the option of purchasing backfiles for journals (Lingle & Robinson, 2009), we found that backfiles were often unavailable and the odds of having backfiles for an unfilled request being unavailable for purchase was 1.5 times higher in 2020 than in 2019. This may indicate an opportunity for publishers to invest in ensuring affordable, persistent access to older articles through backfiles or pay per item options that have not yet been available for libraries or individuals to purchase.

The effects of COVID-19 on library spaces and resources were significant. The widespread closures of physical libraries and limited on-site staffing to fulfill interlibrary loan requests resulted in reduced access to information, particularly for non-English language articles or older articles. While some unfilled requests were available for subscription purchase, this option is not feasible for most libraries. Limited availability of backfiles further compounds the challenge of ensuring access to this information, if large lending research libraries stop lending in DOCLINE, OCLC, or via other mechanisms in the future.

Acknowledgements

We thank all the participating institutions who gave permission for their data to be included in this research; the National Library of Medicine’s Lisa Mohammed, Elisabeth Unger, Gillian Takamaru, and Allison Fisher for consultation on DOCLINE data from participating libraries; and EBSCO Information Services for providing cost data. Finally, we thank our fellow COVID-19 Team collaborators for their guidance and assistance.

References

Allison, A. E., Bryan, B., Franklin, S. G., & Schick, L. C. (2020). Academic health sciences libraries and affiliated hospitals: A conversation about licensing electronic resources. Journal of the Medical Library Association, 108(2). https://doi.org/10.5195/jmla.2020.625

Bahadirli, S., & Sagaltici, E. (2021). Burnout, job satisfaction, and psychological symptoms among emergency physicians during COVID-19 outbreak: A cross-sectional study. Psychiatry and Clinical Psychopharmacology, 31 (1), 67–76. https://doi.org/10.5152/pcp.2021.20180

Carrigan, E. E., Burford, N. G., & Ugaz, A. G. (2018). Collaborative collection management. In S. K. Kendall (Ed.), Health Sciences Collection Management for the Twenty-First Century (pp. 149–166). Rowman & Littlefield.

Collection Access Section, National Library of Medicine. (2018). DOCLINE 2018 MLA update: Network statistics supplement. https://www.nlm.nih.gov/docline/2018_MLA_DOCLINE_Statistics_Supplement.pptx

Creazzo, J., Bakker, C., Koos, J. A., Jo, P., & Alpi, K. M. (2020). Researching interlibrary loan/document delivery usage by health sciences libraries during the COVID-19 pandemic. Journal of Interlibrary Loan, Document Delivery & Electronic Reserve, 29(3–5), 171–179. https://doi.org/10.1080/1072303X.2021.1936739

De Groote, S. L. (2008). Citation patterns of online and print journals in the digital age. Journal of the Medical Library Association, 96(4), 362–369. https://doi.org/10.3163/1536-5050.96.4.012

Foley, D. A., & Tippett, E. (2020). COVID-19 response: The perspectives of infectious diseases physicians and clinical microbiologists. Medical Journal of Australia, 213(9), 431. https://doi.org/10.5694/mja2.50810

Frederick, J., & Wolff-Eisenberg, C. (2020). Academic library strategy and budgeting during the COVID-19 pandemic. Ithaka S+R. https://doi.org/10.18665/sr.314507

Fredericksen, K. (2020). Dream come true or nightmare? The pros and cons of temporarily free electronic resources during COVID-19. Journal of Electronic Resources Librarianship, 32(4), 320–322. https://doi.org/10.1080/1941126X.2020.1822003

Hall, R. A. (2020). Library strategic planning after COVID-19: Don’t fight the last war. College & Research Libraries News, 81(7), 335–336. https://doi.org/10.5860/crln.81.7.335

Harnegie, M. P. (2021). How medical libraries and staff adapt to deliver services during a pandemic. Journal of Hospital Librarianship, 21(2), 173–183. https://doi.org/10.1080/15323269.2021.1904184

Hochstein, C., Han, S., Juneau, P., Takamaru, G., & Unger, E. (2020). Analysis of National Library of Medicine (NLM) DOCLINE® interlibrary loan system request patterns during selected public health events. Journal of Interlibrary Loan, Document Delivery & Electronic Reserve, 29(3–5), 157–170. https://doi.org/10.1080/1072303X.2021.1934216

Howes, L., Ferrell, L., Pettys, G., & Roloff, A. (2021). Adapting to remote library services during COVID-19. Medical Reference Services Quarterly, 40(1), 35–47. https://doi.org/10.1080/02763869.2021.1873616

Huffman, J. (2020). Free kittens? Usage of free library e-resources during the COVID-19 crisis. Journal of Electronic Resources Librarianship, 32(4), 322–328. https://doi.org/10.1080/1941126X.2020.1822005

Knowlton, S. A., Kristanciuk, I., & Jabaily, M. J. (2015). Spilling out of the funnel: How reliance upon interlibrary loan affects access to information. Library Resources & Technical Services, 59(1), Article 1. https://doi.org/10.5860/lrts.59n1.4

Koos, J. A., Scheinfeld, L., & Larson, C. (2021). Pandemic-proofing your library: Disaster response and lessons learned from COVID-19. Medical Reference Services Quarterly, 40(1), 67–78. https://doi.org/10.1080/02763869.2021.1873624

Lemley, T., & Li, J. (2015). “Big deal” journal subscription packages: Are they worth the cost? Journal of Electronic Resources in Medical Libraries, 12(1), 1–10. https://doi.org/10.1080/15424065.2015.1001959

Lingle, V. A., & Robinson, C. K. (2009). Conversion of an academic health sciences library to a near-total electronic library: Part 2. Journal of Electronic Resources in Medical Libraries, 6(4), 279–293. https://doi.org/10.1080/15424060903364750

Lloyd, J. K., Alpi, K. M., Hoogland, M. A., Stephenson, P. L., & Meyer, E. (2022). Interlibrary loan and document delivery in North American health sciences libraries during the early months of the COVID-19 pandemic. Journal of the Medical Library Association, 110(3), 348–357. https://doi.org/10.5195/jmla.2022.1452

Lutz, K. (2020, October 8). Indications of the new normal. Ithaka S+R. https://sr.ithaka.org/blog/indications-of-the-new-normal/

Monroe, K. (n.d.). ARL statistics survey statistical trends. Association of Research Libraries. Retrieved December 4, 2022, from https://www.arl.org/arl-statistics-survey-statistical-trends/

Munson, K. (2012). Herding cats: Challenges in interlibrary loan lending of e-journal articles. Journal of Interlibrary Loan, Document Delivery & Electronic Reserve, 22(3–4), 163–173. https://doi.org/10.1080/1072303X.2012.719599

Nabe, J., & Fowler, D. C. (2015). Leaving the “big deal” … five years later. The Serials Librarian, 69(1), 20–28. https://doi.org/10.1080/0361526X.2015.1048037

National Library of Medicine. (2017). 2017 programs and services. U.S. Department of Health and Human Services. https://www.nlm.nih.gov/ocpl/anreports/fy2017.pdf

O’Brien, K. (2004). Cancelled requests: A study of interlibrary loan requests for items held in a medical library collection. Medical Reference Services Quarterly, 23(3), 17–23. https://doi.org/10.1300/J115v23n03_02

Ohler, L. A., & Pitts, J. (2021). From peril to promise: The academic library post–COVID-19. College & Research Libraries News, 82(1), 41. https://doi.org/10.5860/crln.82.1.41

Research Solutions. (n.d.). Reprints Desk. Retrieved December 10, 2022, from https://www.researchsolutions.com/about

Schonfield, R. C. (2020, April 21). Research library digitization has found its moment. Ithaka S+R. https://sr.ithaka.org/blog/research-library-digitization-has-found-its-moment/

Sharing and Transforming Access to Resources Section International Interlibrary Loan Committee. (2009). Lending and borrowing across borders: Issues and challenges with international resource sharing. Reference & User Services Quarterly, 49(1), 54–63.

Thompson, J., Kearney, I., Jackson, R., Rinne, J., & Adams, L. (2021). The state of medical education and practice (SoMEP) barometer survey 2021 (p. 51). General Medical Council. https://www.gmc-uk.org/-/media/documents/somep-2021-barometer-report_pdf-88506203.pdf?la=en&hash=E2E929F9DB51064E05DCFFED56FAEA5726160453

Tsay, M. (1999). Library journal use and citation age in medical science. Journal of Documentation, 55(5), 543–555. https://doi.org/10.1108/EUM0000000007158

Walpole, S. C. (2019). Including papers in languages other than English in systematic reviews: Important, feasible, yet often omitted. Journal of Clinical Epidemiology, 111, 127–134. https://doi.org/10.1016/j.jclinepi.2019.03.004

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.