Evidence Based Library and Information Practice

Using Evidence in Practice

 

Using Data to Break Apart Journal Packages

 

Alison Ambi

Science Research Liaison Librarian

Memorial University of Newfoundland

St. John’s, Newfoundland & Labrador, Canada

Email: aambi@mun.ca

 

Erin Alcock

Science Research Liaison Librarian

Memorial University of Newfoundland

St. John’s, Newfoundland & Labrador, Canada

Email: ekalcock@mun.ca

 

Pamela Morgan

Information Resources Librarian

Memorial University of Newfoundland

St. John’s, Newfoundland & Labrador, Canada

Email: pmorgan@mun.ca

 

Amanda Tiller-Hackett

Humanities Collection Development Librarian

Memorial University of Newfoundland

St. John’s, Newfoundland & Labrador, Canada

Email: atiller@mun.ca

 

Received: 4 Nov. 2016    Accepted: 14 Nov. 2016

 

 

cc-ca_logo_xl 2016 Ambi, Alcock, Morgan, and Tiller-Hackett. This is an Open Access article distributed under the terms of the Creative Commons‐Attribution‐Noncommercial‐Share Alike License 4.0 International (http://creativecommons.org/licenses/by-nc-sa/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one.

 

 

Setting

 

Memorial University of Newfoundland is the only university in Newfoundland and Labrador and has approximately 20,000 students. The university aims to provide both a comprehensive undergraduate program and research-intensive graduate programs. Because of the university’s geographic isolation it is particularly important for the library to provide a comprehensive collection. Unlike in larger centres, researchers can not avail themselves of the resources held by nearby institutions. We therefore tend to adopt a cautious and conservative approach to cancellations or weeding.

 

Problem

 

Over the past decade there has been a steady increase in journal subscription prices while the materials budgets of academic libraries has remained fixed or even decreased. When this trend coincided with a precipitous devaluation of the Canadian dollar beginning in 2013, many Canadian academic libraries were thrust into a serials budget crisis.

 

Librarians at Memorial had implemented many smaller cost-saving measures during 2013 and 2014, however, it became evident that trimming the low-hanging fruit would not suffice. Librarians needed to set their sights on significantly larger cancellation targets to make ends meet. A committee was formed to evaluate our cancellation options.

 

Based on an analysis of the target savings, the subscriptions available for renewal at the end of 2015, and strategies to avoid disadvantaging any particular discipline, the committee recommended cancelling four large multi-disciplinary publisher journal packages – Cambridge, Oxford, Springer, and Wiley – and buying back only the most crucial titles. This was a dramatic proposal by any standard, but especially for Memorial. It was, however, unavoidable.

 

Evidence

 

To determine which journals were critical for supporting our researchers and students, we gathered cost, use, and alternate access data for each of the 4,231 titles in the 4 packages, and we solicited feedback from the university community. We chose not to consider impact factors in our analysis because of the flaws and limitations of this metric.

 

Pinning down a definitive title list and compiling use data was challenging for a number of reasons. Title lists are in constant flux due to publisher mergers and acquisitions, new journals launched by publishers annually, journals that change names, merge, split into two, or are discontinued, journals that become open access, and the tendency of society journals to move from one publisher to another. The sub-packages within some of these “big deal” packages presented another challenge, since there were some journals which were not priced individually and could not be purchased individually.

 

Community Feedback Data

 

In June 2015 we conducted a survey of the university community. We informed respondents about the plan to cancel the four publisher packages and the rationale for doing so. We asked the respondents to provide lists of journals they deemed “essential” for their research, “important” for their research, and those that they needed for their students and/or teaching activities.

 

Approximately 16% of the faculty/researchers on campus responded to the survey. Of the 1,224 titles mentioned in the survey, approximately 1/3 were in one of the 4 packages under review.  The combined list price of the 1,224 journals was significantly more than the cost of the 4 packages, so we knew that we would not be able to retain them all.

 

Of the Cambridge, Oxford, Wiley, and Springer titles that were not mentioned in the survey, 88 showed significant use and had over 200 annual article downloads. There were 3 titles with JR1 averages in the vicinity of 1000 which had not been mentioned in the survey. At the opposite end of the scale, 17 titles showing JR1 averages of 0 were mentioned by survey respondents.

 

Given the low response rate and the fact that some very high-use journals had not been mentioned, it was clear that we could not rely heavily on the survey data to make our decisions.

 

Journal Metrics

 

Cost Data: When we started our analysis the publishers had not yet released their 2016 price lists. We therefore approximated 2016 prices by estimating a 5% increase over the 2015 list prices.

 

Use Data: COUNTER data was available from all four publishers to some extent. For each title we calculated:

 

  1. Average of JR1[1] over the past 3 years
  2. Average of JR5[2] over the past 3 years (where possible)

 

Alternate Access Data: We conducted an overlap analysis between each publisher package and our aggregator subscriptions to identify titles with alternate availability and the embargo lengths for each title. Just over 50% of the journals in the 4 packages were available with a 12 month embargo or better.

 

Combining Cost, Use, and Alternate Access Data:

 

  1. We calculated the cost per download
  2. For journals to which we had embargoed access we wanted to establish a threshold for current year use at which a subscription might be more cost effective than relying on document delivery. Using a cost estimate of $30 to provide a single article through our document delivery service, and estimating that the volume of document delivery requests would not exceed 25% of JR5, we identified journals for which the cost of current year document delivery requests was at risk of exceeding the cost of purchase. Our choice of a 4:1 ratio was an over-estimate. Subsequently published literature has suggested that the ratio is likely to be considerably lower. Nabe and Fowler (2015) estimate the ratio at 10:1 and according to Scott (2016) it might be as low as 17:1.

 

 

Table 1

Summary of Variables Calculated for Each Journal Title

Variable Name

Method of Calculation

JR1 Average

[(JR1 2012) +(JR1 2013)+(JR1 2014)]/3

JR5 Average

[(JR1 2012) +(JR1 2013)+(JR1 2014)]/3

Cost

(2015 list price in USD) X (5% price increase) X (currency conversion factor) X (tax factor)

Cost/Download

(cost) / (JR1 average)

Best Embargo

The shortest embargo available in our aggregator products

Estimated Cost of Document Delivery

Our institution uses $30 as an estimate of the cost to process a single document delivery request

Anticipated volume of current- year document delivery requests

(JR5) X (0.25)

Anticipated Document Delivery Burden

[(JR5 Average) X (0.25) X ($30)] – cost    

 

(If this was greater than zero for journals available with a 12-month embargo there was a risk that the cost of anticipated document delivery requests would exceed the cost of subscription)

 

 

Implementation

 

By setting thresholds for use, cost/download, and volume of community feedback and considering the availability of embargoed access, we were able to use Microsoft Excel to filter out a list of the most crucial titles. This short initial list had a combined list price well within what we could afford to spend. Information about the proposed fate of all journals in the packages, and about alternate access for the titles slated for cancellation was distributed to the university community for a second round of feedback.

 

We received many requests from faculty to reverse cancellation decisions. We revisited the cost, use, and alternate access data for the requested titles in the context of the qualitative information provided by researchers. To the extent possible, we added those titles for which a case could reasonably be made.

 

Outcome

 

Of the 4,231 titles in the 4packages, we retained subscriptions to 220 (5%), saving a total of approximately $800,000 when compared to what we would have paid had we renewed the 4 packages. 

 

Of the remaining titles, 1,942 (49%) were available with a 1 year embargo in our aggregator products, and 75 (2%) were available with no embargo. Removing the handful of titles that had either become open access, moved to a different publisher, discontinued publications, etc., the final number of journals that we truly cancelled with no alternate access options was 1,758 (44%). See Figure 1.

 

 

Figure 1

Final buy-back results.

 

 

Of the 1,758 cancelled journals with no alternate access, many had very low use. For 202 titles (11%) there had been no article downloads in the past three years (JR1 Average = 0), and for 913 titles (51%) there were 10 or fewer article downloads per year averaged over the past 3 years (JR1 Average <=10).

 

There were journals we did not want to cancel and we are watching the document delivery statistics for these titles closely. It has not yet been a full year since the cancellations, but an initial analysis of our document delivery data has not revealed any significant demand for articles from any of the cancelled journals.

 

From a purely cost-saving perspective our process was successful. The data we gathered and how we used it to make our decisions reduced our subscription expenditure in 2016 and allowed us to avoid increased document delivery costs. We succeeded in dramatically decreasing our journal subscriptions while minimizing the impact on the university community.

 

Reflection

 

Although we succeeded in saving enough money, we acknowledge that from other perspectives, the outcome of this process was not ideal.

 

Document delivery can provide researchers with access to any of the content we have cancelled, but we have lost easy and rapid access to content that might have been useful to our community. The literature indicates that there is little impact on document delivery when a big deal is cancelled (Calvert, Gee, Malliett, & Flemming, 2013; McGrath, 2012). Some students and researchers may be deterred from accessing this content using document delivery. Others will turn to illegal or underground sources like Sci-Hub and peer networks which will skew the data, making future subscription decisions more difficult.

 

Volume of use is only one way to measure the relative value of a journal within an institution, and it has several flaws and limitations. Our numbers game inherently favoured the research interests of larger departments and research groups on the campus, journals with interdisciplinary appeal, and journals in disciplines where the primary research literature is more heavily used by undergraduate students. The value of journals representing emerging research directions, smaller departments and research groups, and disciplines in which primary literature is typically accessed only by graduate students and faculty, cannot easily be judged on the basis of use.

 

Faculty tend to use very different criteria from librarians when assigning value to journals, and in some disciplines they care deeply about impact factors. User-perceived criteria are often personal (“I have published in this journal”) or prestige-based (“it would be embarrassing for the institution if we cancelled this famous journal”) and consequently more emotionally determined and less susceptible to arguments based on use and cost. As a result, many of those engaged in research and teaching at our institution feel that resources of high value to them have been cancelled. It might have been possible to bring the outcome of our analyses closer to faculty perceptions by systematically including data about local publishing activities.

 

Subscribing individually to the journals we chose to retain has also placed a significant burden on our acquisitions department and complicated the process of making the journals available through our discovery layer. The impact of this process on less tangible institutional resources has been significant, especially if one includes the time devoted by the cancellations committee.

 

On a more positive note, there is generally a more comprehensive understanding among the faculty and administration at our institution of trends in academic journal publishing and the consequences for academic libraries. The cancellations project provided many opportunities for librarians to discuss these issues with our colleagues in academia.

 

Conclusion

 

The librarians engaged in the cancellations process learned an enormous amount about the complexities of academic journal publishing. In addition, the scale of the project, which involved simultaneously grappling with four packages, forced us to learn a lot about data manipulation and analysis. This has enabled us to approach our next task - the analysis of the Taylor & Francis package in 2016 - with more efficiency and nuance.

 

When we embarked on this project in 2015, we knew that many other Canadian libraries would be forced to cancel “Big Deals”, but few had publicized their cancellation activities. Because we were pioneers of a sort, it was difficult in 2015 to convince our community that the situation at Memorial was not particularly unique. Now that many other Canadian institutions have publicized their cancellation efforts, our faculty are more conversant with the issues in academic journal publishing, there is a heightened awareness of the budgetary crisis in our province, and our review process in 2016 is progressing with less passionate opposition. We hope that documenting our experiences will be of some benefit to other librarians faced with this task.

 

We will continue to monitor document delivery requests for cancelled titles, as well as the use data for titles that were retained to determine if annual adjustments to our subscription list is warranted. For the majority of the cancelled titles we have perpetual access to issues published during the years to which we subscribed to the packages. It is therefore not surprising that we have not yet observed any increases in document delivery requests.

 

Now that we have broken free from several publisher packages we have the flexibility to compare the relative values of individual journals without the need to consider the publisher.

 

There is no doubt that the “big deal” package is the most cost-effective and efficient way to subscribe to the journals from many academic publishers. However, no matter how cost effective it is, if the cost is higher than libraries can afford, we will have no choice but to continue breaking the packages apart.

 

References

 

Calvert, K., Gee, W., Malliett, J., & Flemming, R. (2013). Is ILL enough? Examining ILL demand after journal cancellations at three North Carolina universities. Proceedings of the Charleston Library Conference, Charleston, South Carolina. http://dx.doi.org/10.5703/1288284315297

 

McGrath, M. (2012). Fighting back against the big deals: A success story from the UK. Interlending & Document Supply, 40(4), 178-186. http://dx.doi.org/10.1108/02641611211283831

 

Nabe, J., & Fowler, D. C. (2015). Leaving the “big deal” … five years later. The Serials Librarian, 69(1), 20-28. http://dx.doi.org/10.1080/0361526X.2015.1048037

 

Scott, M. (2016). Predicting use: COUNTER usage data found to be predictive of ILL use and ILL use to be predictive of COUNTER use. The Serials Librarian, 71(1), 20-24. http://dx.doi.org/10.1080/0361526X.2016.1165783

 

 



[1] JR1 for a journal = the total number of articles downloaded during one year

[2] JR5 for a journal = the number of articles downloaded in a given year that were published in that same year






Evidence Based Library and Information Practice (EBLIP) | EBLIP on Twitter