Using Evidence in Practice
Using Data to
Break Apart Journal Packages
Alison
Ambi
Science
Research Liaison Librarian
Memorial
University of Newfoundland
St.
John’s, Newfoundland & Labrador, Canada
Email:
aambi@mun.ca
Erin
Alcock
Science
Research Liaison Librarian
Memorial
University of Newfoundland
St.
John’s, Newfoundland & Labrador, Canada
Email:
ekalcock@mun.ca
Pamela
Morgan
Information
Resources Librarian
Memorial
University of Newfoundland
St.
John’s, Newfoundland & Labrador, Canada
Email:
pmorgan@mun.ca
Amanda
Tiller-Hackett
Humanities
Collection Development Librarian
Memorial
University of Newfoundland
St.
John’s, Newfoundland & Labrador, Canada
Email:
atiller@mun.ca
Received: 4 Nov. 2016 Accepted: 14 Nov. 2016
2016 Ambi,
Alcock, Morgan, and Tiller-Hackett. This is an Open Access article distributed
under the terms of the Creative Commons‐Attribution‐Noncommercial‐Share Alike
License 4.0 International (http://creativecommons.org/licenses/by-nc-sa/4.0/),
which permits unrestricted use, distribution, and reproduction in any medium,
provided the original work is properly attributed, not used for commercial
purposes, and, if transformed, the resulting work is redistributed under the
same or similar license to this one.
Setting
Memorial University of Newfoundland is the
only university in Newfoundland and Labrador and has approximately 20,000
students. The university aims to provide both a comprehensive undergraduate
program and research-intensive graduate programs. Because of the university’s
geographic isolation it is particularly important for the library to provide a
comprehensive collection. Unlike in larger centres, researchers can not avail
themselves of the resources held by nearby institutions. We therefore tend to
adopt a cautious and conservative approach to cancellations or weeding.
Problem
Over the past decade there has been a steady
increase in journal subscription prices while the materials budgets of academic
libraries has remained fixed or even decreased. When this trend coincided with
a precipitous devaluation of the Canadian dollar beginning in 2013, many
Canadian academic libraries were thrust into a serials budget crisis.
Librarians at Memorial had implemented many
smaller cost-saving measures during 2013 and 2014, however, it became evident
that trimming the low-hanging fruit would not suffice. Librarians needed to set
their sights on significantly larger cancellation targets to make ends meet. A
committee was formed to evaluate our cancellation options.
Based on an analysis of the target savings,
the subscriptions available for renewal at the end of 2015, and strategies to
avoid disadvantaging any particular discipline, the committee recommended
cancelling four large multi-disciplinary publisher journal packages –
Cambridge, Oxford, Springer, and Wiley – and buying back only the most crucial
titles. This was a dramatic proposal by any standard, but especially for
Memorial. It was, however, unavoidable.
Evidence
To determine which journals were critical for supporting
our researchers and students, we gathered cost, use, and alternate access data
for each of the 4,231 titles in the 4 packages, and we solicited feedback from
the university community. We chose not to consider impact factors in our
analysis because of the flaws and limitations of this metric.
Pinning down a definitive title list and
compiling use data was challenging for a number of reasons. Title lists are in
constant flux due to publisher mergers and acquisitions, new journals launched
by publishers annually, journals that change names, merge, split into two, or
are discontinued, journals that become open access, and the tendency of society
journals to move from one publisher to another. The sub-packages within some of
these “big deal” packages presented another challenge, since there were some
journals which were not priced individually and could not be purchased
individually.
Community Feedback Data
In June 2015 we conducted a survey of the
university community. We informed respondents about the plan to cancel the four
publisher packages and the rationale for doing so. We asked the respondents to
provide lists of journals they deemed “essential” for their research,
“important” for their research, and those that they needed for their students
and/or teaching activities.
Approximately 16% of the faculty/researchers
on campus responded to the survey. Of the 1,224 titles mentioned in the survey,
approximately 1/3 were in one of the 4 packages under review. The combined list price of the 1,224 journals
was significantly more than the cost of the 4 packages, so we knew that we
would not be able to retain them all.
Of the Cambridge, Oxford, Wiley, and Springer
titles that were not mentioned in the survey, 88 showed significant use and had
over 200 annual article downloads. There were 3 titles with JR1 averages in the
vicinity of 1000 which had not been mentioned in the survey. At the opposite
end of the scale, 17 titles showing JR1 averages of 0 were mentioned by survey
respondents.
Given the low response rate and the fact that
some very high-use journals had not been mentioned, it was clear that we could
not rely heavily on the survey data to make our decisions.
Journal Metrics
Cost Data: When we
started our analysis the publishers had not yet released their 2016 price
lists. We therefore approximated 2016 prices by estimating a 5% increase over
the 2015 list prices.
Use Data: COUNTER
data was available from all four publishers to some extent. For each title we
calculated:
Alternate Access Data:
We conducted an overlap analysis between each publisher package and our
aggregator subscriptions to identify titles with alternate availability and the
embargo lengths for each title. Just over 50% of the journals in the 4 packages
were available with a 12 month embargo or better.
Combining Cost, Use, and Alternate Access Data:
- We calculated the cost per download
- For journals to which we had embargoed access we wanted to
establish a threshold for current year use at which a subscription might
be more cost effective than relying on document delivery. Using a cost
estimate of $30 to provide a single article through our document delivery
service, and estimating that the volume of document delivery requests
would not exceed 25% of JR5, we identified journals for which the cost of
current year document delivery requests was at risk of exceeding the cost
of purchase. Our choice of a 4:1 ratio was an over-estimate. Subsequently
published literature has suggested that the ratio is likely to be
considerably lower. Nabe and Fowler (2015) estimate the ratio at 10:1 and according to Scott (2016) it might be as low as 17:1.
Table 1
Summary of Variables Calculated for Each
Journal Title
|
Variable Name |
Method of Calculation |
|
JR1 Average |
[(JR1 2012) +(JR1 2013)+(JR1 2014)]/3 |
|
JR5 Average |
[(JR1 2012) +(JR1 2013)+(JR1 2014)]/3 |
|
Cost |
(2015 list price in USD) X (5% price
increase) X (currency conversion factor) X (tax factor) |
|
Cost/Download |
(cost) / (JR1 average) |
|
Best Embargo |
The shortest embargo available in our
aggregator products |
|
Estimated Cost of Document Delivery |
Our institution uses $30 as an estimate of
the cost to process a single document delivery request |
|
Anticipated volume of current- year document
delivery requests |
(JR5) X (0.25) |
|
Anticipated Document Delivery Burden |
[(JR5 Average) X (0.25) X ($30)] – cost (If this was greater than zero for journals
available with a 12-month embargo there was a risk that the cost of
anticipated document delivery requests would exceed the cost of subscription) |
Implementation
By setting thresholds for use, cost/download,
and volume of community feedback and considering the availability of embargoed
access, we were able to use Microsoft Excel to filter out a list of the most
crucial titles. This short initial list had a combined list price well within
what we could afford to spend. Information about the proposed fate of all
journals in the packages, and about alternate access for the titles slated for
cancellation was distributed to the university community for a second round of
feedback.
We received many requests from faculty to
reverse cancellation decisions. We revisited the cost, use, and alternate
access data for the requested titles in the context of the qualitative
information provided by researchers. To the extent possible, we added those
titles for which a case could reasonably be made.
Outcome
Of the 4,231 titles in the 4packages, we
retained subscriptions to 220 (5%), saving a total of approximately $800,000
when compared to what we would have paid had we renewed the 4 packages.
Of the remaining titles, 1,942 (49%) were
available with a 1 year embargo in our aggregator products, and 75 (2%) were
available with no embargo. Removing the handful of titles that had either
become open access, moved to a different publisher, discontinued publications,
etc., the final number of journals that we truly cancelled with no alternate
access options was 1,758 (44%). See Figure 1.
Figure 1
Final buy-back
results.
Of the 1,758 cancelled journals with no
alternate access, many had very low use. For 202 titles (11%) there had been no
article downloads in the past three years (JR1 Average = 0), and for 913 titles
(51%) there were 10 or fewer article downloads per year averaged over the past
3 years (JR1 Average <=10).
There were journals we did not want to cancel
and we are watching the document delivery statistics for these titles closely.
It has not yet been a full year since the cancellations, but an initial
analysis of our document delivery data has not revealed any significant demand
for articles from any of the cancelled journals.
From a purely cost-saving perspective our
process was successful. The data we gathered and how we used it to make our
decisions reduced our subscription expenditure in 2016 and allowed us to avoid increased
document delivery costs. We succeeded in dramatically decreasing our journal
subscriptions while minimizing the impact on the university community.
Reflection
Although we succeeded in saving enough money,
we acknowledge that from other perspectives, the outcome of this process was
not ideal.
Document delivery can provide researchers with
access to any of the content we have cancelled, but we have lost easy and rapid
access to content that might have been useful to our community. The literature
indicates that there is little impact on document delivery when a big deal is
cancelled (Calvert, Gee, Malliett, & Flemming, 2013;
McGrath, 2012). Some students and researchers may be deterred from accessing this
content using document delivery. Others will turn to illegal or underground
sources like Sci-Hub and peer networks which will skew the data, making future
subscription decisions more difficult.
Volume of use is only one way to measure the
relative value of a journal within an institution, and it has several flaws and
limitations. Our numbers game inherently favoured the research interests of
larger departments and research groups on the campus, journals with
interdisciplinary appeal, and journals in disciplines where the primary
research literature is more heavily used by undergraduate students. The value
of journals representing emerging research directions, smaller departments and
research groups, and disciplines in which primary literature is typically
accessed only by graduate students and faculty, cannot easily be judged on the
basis of use.
Faculty tend to use very different criteria
from librarians when assigning value to journals, and in some disciplines they
care deeply about impact factors. User-perceived criteria are often personal
(“I have published in this journal”) or prestige-based (“it would be
embarrassing for the institution if we cancelled this famous journal”) and
consequently more emotionally determined and less susceptible to arguments
based on use and cost. As a result, many of those engaged in research and
teaching at our institution feel that resources of high value to them have been
cancelled. It might have been possible to bring the outcome of our analyses
closer to faculty perceptions by systematically including data about local
publishing activities.
Subscribing individually to the journals we
chose to retain has also placed a significant burden on our acquisitions
department and complicated the process of making the journals available through
our discovery layer. The impact of this process on less tangible institutional
resources has been significant, especially if one includes the time devoted by
the cancellations committee.
On a more positive note, there is generally a
more comprehensive understanding among the faculty and administration at our
institution of trends in academic journal publishing and the consequences for
academic libraries. The cancellations project provided many opportunities for
librarians to discuss these issues with our colleagues in academia.
Conclusion
The librarians engaged in the cancellations
process learned an enormous amount about the complexities of academic journal
publishing. In addition, the scale of the project, which involved
simultaneously grappling with four packages, forced us to learn a lot about
data manipulation and analysis. This has enabled us to approach our next task -
the analysis of the Taylor & Francis package in 2016 - with more efficiency
and nuance.
When we embarked on this project in 2015, we
knew that many other Canadian libraries would be forced to cancel “Big Deals”,
but few had publicized their cancellation activities. Because we were pioneers
of a sort, it was difficult in 2015 to convince our community that the
situation at Memorial was not particularly unique. Now that many other Canadian
institutions have publicized their cancellation efforts, our faculty are more
conversant with the issues in academic journal publishing, there is a
heightened awareness of the budgetary crisis in our province, and our review
process in 2016 is progressing with less passionate opposition. We hope that
documenting our experiences will be of some benefit to other librarians faced
with this task.
We will continue to monitor document delivery
requests for cancelled titles, as well as the use data for titles that were
retained to determine if annual adjustments to our subscription list is
warranted. For the majority of the cancelled titles we have perpetual access to
issues published during the years to which we subscribed to the packages. It is
therefore not surprising that we have not yet observed any increases in
document delivery requests.
Now that we have broken free from several
publisher packages we have the flexibility to compare the relative values of
individual journals without the need to consider the publisher.
There is no doubt that the “big deal” package
is the most cost-effective and efficient way to subscribe to the journals from
many academic publishers. However, no matter how cost effective it is, if the
cost is higher than libraries can afford, we will have no choice but to
continue breaking the packages apart.
References
Calvert, K., Gee, W., Malliett, J., & Flemming,
R. (2013). Is ILL enough? Examining ILL demand after journal cancellations at
three North Carolina universities. Proceedings of the Charleston Library
Conference, Charleston, South Carolina. http://dx.doi.org/10.5703/1288284315297
McGrath, M. (2012). Fighting back against the big
deals: A success story from the UK. Interlending & Document Supply, 40(4),
178-186. http://dx.doi.org/10.1108/02641611211283831
Nabe, J., & Fowler, D. C. (2015). Leaving the
“big deal” … five years later. The Serials Librarian, 69(1), 20-28. http://dx.doi.org/10.1080/0361526X.2015.1048037
Scott, M. (2016). Predicting use: COUNTER usage
data found to be predictive of ILL use and ILL use to be predictive of COUNTER
use. The Serials Librarian, 71(1), 20-24. http://dx.doi.org/10.1080/0361526X.2016.1165783