A Re-examination of Online Journal Quality and Investigation of the Possible Impact of Poor Electronic Surrogate Quality on Researchers

Abstract

Objective – This study re-examines the findings of a paper (Ladd, 2010) that investigated whether evidence indicated print equivalent journal collections needed to be preserved, based on the quality of their electronic surrogates. The current study investigates whether: 1) electronic surrogate articles that failed (i.e., the print equivalent article needed to be consulted to view all the content/information) in the first study had improved in quality; and 2) there was evidence that poor-quality electronic surrogates could impact on research if the print equivalent articles did not exist.

Methods – Each of the 198 PDF documents identified in the 2010 study as failing were re-examined to assess whether any change in quality had occurred. To assess the possible impact for researchers if they needed to rely solely on poor-quality electronic journal surrogates, citation data were collected for each of the failed scholarly PDFs using Web of Science and Scopus, and usage count data were collected from Web of Science.

Results – Across the electronic journal backfiles/archives examined, there were 13.6% fewer failures of electronic surrogates for all PDF documents than in the original study, while for scholarly PDF documents (e.g., research papers) there were 13.8% fewer failures. One electronic journal archive accounted for 91.7% of the improvement for scholarly PDF documents. A second archive accounted for all the observed improvement for non-scholarly PDF documents. The study found that for the failed scholarly PDF documents from the original study, 58.7% had been cited or had Web of Science usage counts from 2010 onward.

Conclusion – The study demonstrates a continued need for retaining print equivalent journal titles for the foreseeable future, while poor-quality electronic surrogates are being replaced and digitally preserved. There are still poor-quality images, poor-quality scans of text-only articles, missing pages, and even content of PDF documents that could not be explained (e.g., incorrect text for images when compared to the print). While it is known that not all researchers will consult each of the papers that they cite, although it is best practice to do so, the extent of citations of the failed scholarly PDF documents indicate that having to rely solely on electronic surrogates could pose a problem for researchers.