PROGRAM DESCRIPTION / DESCRIPTION DU PROGRAMME An Education Session Developed in Response to Low Health Professional Awareness of Predatory Journals

Predatory journals have arisen as a significant problem within scholarly literature and efforts to raise awareness about them have focused on researchers looking to publish their work.  Because health professionals use scholarly literature to inform their practice, develop policy, and conduct their own research, they should be familiar with the harmfulness of predatory journals.  This paper describes an education session delivered to health professionals throughout the province of X.  Developed in response to health professionals’ low awareness of predatory journals, it was designed to raise awareness and to provide tools and techniques to critically appraise the scholarly literature.


Introduction
Predatory journals have recently become a significant problem within scholarly literature [1][2][3][4]. Though there is no single unified definition of predatory journals, they may be characterised as journals that claim to uphold scholarly publishing standards while neglecting to meet those standards; in particular, predatory journals claim to engage in rigorous peer review while conducting only a cursory review or not conducting a peer review at all [5]. Recognizing that a definition of predatory journals is elusive [5], for the purposes of this paper readers are encouraged to consider that predatory journals exist on a spectrum, where there is not necessarily a hard line to distinguish predatory journals from legitimate scholarly journals. Education about predatory journals is therefore an attempt to raise awareness of traits that may contribute to the "predatoryness" of a journal rather than attempt to define predatory journals rigidly. Efforts to raise awareness about predatory journals have focused on the academic community and have been directed at researchers looking to publish their work in academic journals [6,7]. Few publications have addressed the awareness that health professionals have about predatory journals, though the literature suggests that awareness among health professionals is low [8,9]. Since health professionals use scholarly literature to inform decisions, and are authors of it, there is a risk that health professionals could use the content to inform their decisions, or publish in predatory journals unknowingly. Thus, there is a need for health professionals to be aware of the existence and problems associated with predatory journals so that they can make fully informed decisions with regard to their publication and practice.
Health librarians have a role to play in active education about predatory journals [5,10]. Though they may be engaged in such education, the details have not been discussed in the scholarly literature. This paper describes an education session delivered to health professionals to raise their awareness of predatory journals, and to provide them with the tools and techniques to critically appraise scholarly literature.

Background
Manitoba's Health Information and Knowledge Network (MHIKNET) is an outreach program of the University of Manitoba Libraries. MHIKNET provides library services to physicians throughout the province, all health professionals working in regional health authorities throughout rural Manitoba, and all staff of Manitoba Health, Seniors, and Active Living. These health professionals use scholarly literature to inform their practice, develop policy, and conduct research. In the summer of 2017, MHIKNET librarians began providing in-person and online education sessions, with a focus on information literacy and critical appraisal [11]. Each session began with an overview of MHIKNET's services. During the presentation, presenters polled participants between 2 and 5 times to assess their familiarity with, and comprehension of, the material. After each session, registrants and participants were emailed copies of the presentation, a summary handout, and a link to an online assessment survey.
One education session focused on improving Google search techniques and critically evaluating information. During this session participants were polled about their familiarity with predatory journals, then were provided a brief 10-minute introduction to predatory journals, describing what they are, why they are problematic, and techniques to assess content within scholarly publications. One hundred participants attended the Google sessions, 35 in person and 65 online, with 23 participants (35%) from the online session responding to a poll asking if they had heard of predatory journals. A majority of poll respondents indicated either that they had not heard of them (n=20, 87%), or that they had heard the term but were unsure what it meant (n=2, 8.7%). Only one participant (4.3%) indicated they were familiar with predatory journals. Poll results from the in-person session were not retained. Following the sessions, participants were asked to complete an online assessment survey, and it was returned with a 51% response rate (n=51). One question inquired as to how valuable the brief predatory journal section of the session was. Thirty-three participants (64.7%) indicated they "learned a lot," twelve (23.5%) indicated they "learned a little bit," only four participants (7.8%) indicated they "already knew about" predatory journals, and two participants (3.9%) did not answer the question. Despite the small sample, MHIKNET librarians believed this indicated an unmet need for health professionals. Health professionals rely on the information found in scholarly literature, and MHIKNET librarians believed it necessary to alert them to the degree that information could be unreliable, and to the tools needed to appraise the scholarly literature.
To address this identified need, an hour-long education session was developed for MHIKNET clients, devoted to education about predatory journals. At the beginning of this new session, participants were asked about their pre-existing levels of knowledge regarding predatory journals and at its conclusion, they were asked what they had learned.

Literature Review
Predatory journals are named for their propensity to "prey" on scholars hoping to publish in legitimate academic journals. These journals mimic the outward appearance of online scholarly journals. Librarian Jeffrey Beall first applied the term "predatory" to these journals and their respective publishers in 2010 [12]. Beall maintained a list of alleged predatory publishers that was shuttered in early 2017, and while he was the most well-known name in discussions about predatory publishing, he drew heavy controversy for his lack of transparency in assigning the "predatory" designation, his apparent bias against the open access movement, and his focus on traits that targeted journals from the Global South [13][14][15]. The term "predatory" itself has been questioned for a variety of reasons, including that some scholars likely publish in these journals with the knowledge that they are not authoritative peer-reviewed publications, or because non-malicious, but poorquality, publications may be unfairly grouped with the predatory journals. Alternative nomenclature has been suggested for these journals, such as "deceptive," "bad faith," or "pseudo-journals," though none of these terms have gained widespread popularity or familiarity, and as such, "predatory journals" will be used throughout this paper [5,13,16].
Recent years have seen warnings from the scholarly community and health organizations, including the International Academy of Nursing Editors (INANE), on the existence and nature of predatory journals, and how they can clog the literature with useless information [17][18][19]. This problem is amplified when researchers view these papers in databases like Google Scholar or PubMed, without contextual consideration of the journal they are published in, which can lead to accidental citing of poor quality materials [2,8,[20][21][22]. There is currently no mechanism for journal editors to efficiently check reference lists for predatory publications, and peer review consideration of reference lists may be insufficient [23,24]. The amount of material published in predatory journals, combined with the potential of legitimate researchers to cite these publications, undermines the credibility of scholarly literature [25]. Numerous sting operations have demonstrated that these journals will accept articles with minimal or no peer review [26][27][28]. While most papers submitted to predatory journals are not as outlandish as those submitted in sting operations, studies indicate that their quality is lacking [3,29].
Though the extent of the effects of material published in predatory journals is unclear, some academic fields have reported disruptions caused by these publications. For example, Ragahavan et al. discuss the detrimental effect of predatory publications on ichthyologic taxonomy [30]. In medicine it has been speculated that as many as 50% of new case-report journals are predatory and that there are comparable numbers of predatory journals in neuroscience and neurology as legitimate ones [1,19,31,32]. Many warnings advise authors on how to avoid publishing in predatory journals, and initiatives such as Think, Check, Submit, provide tools and resources to help academics in this regard [7,20,33]. Comparatively, there has been little done to create awareness of predatory journals for those who use scholarly articles in their practice, but do not necessarily publish in scholarly journals.
Predatory journals are becoming more common, though it is difficult to know how many there are. The number of predatory journals has risen markedly since 2010 and is growing at an exponential rate [34][35][36][37]. For example, Oermann et al. found 140 predatory nursing journals from 75 publishers [36]. Though the percentage of predatory journals per subject disciplines is unknown, they still represent only a small portion of publications; for example, Perlin et al. found that the number of articles written by Brazilian researchers and published in predatory journals tripled between 2010 and 2015, but that they still accounted for fewer than 2% of articles written by Brazilian researchers overall [35]. Despite the research into specific disciplines and countries, the overall prevalence of predatory journals within the scholarly literature is not known and can only be speculated upon.
Most predatory journals are not indexed in library databases [38], however, some have been found in ProQuest Central, EBSCO Academic Search Complete, Gale Academic OneFile, Scopus, and have appeared in some databases more than others, including PubMed [1,32,34,39,40]. Predatory journals are also found in the Directory of Open Access Journals (DOAJ) [38,39,41] and are frequently included in Google Scholar, likely because it is a web crawler, not an index [21,37,41]. The DOAJ has taken steps to reduce number of predatory journals in its listing [42].
Predatory journals have the capacity to undermine the overall credibility of scholarship, and raising awareness of them is critical to limiting that impact. Researchers in the sciences, and health sciences particularly, must be aware of predatory journals, as these areas contain them in the highest concentration [39]. Health professionals use academic literature in their day-to-day practice, in policy development, within their own research, and as a vehicle to publish that research. Since health professionals may lack direct access to the subscription databases purchased by academic institutions, they may rely on freely accessible databases such as PubMed, Google Scholar, and the DOAJthat is, those databases more likely to contain predatory journals [1,40,41]. Health professionals' lack of awareness of predatory journals could have deleterious impacts on patient care and health policy development. To date, the lone study of health professionals' awareness of predatory journals found that awareness to be low; medical and veterinary authors who attended a scholarly writing workshop lacked an understanding of why some journals were more trustworthy than others, and only 24% had heard of predatory journals [43].

Description
In response to the identified low levels of awareness about predatory journals, a one-hour education session for MHIKNET clients was developed titled The Perils of Predatory Journals. This session was presented twice, once in person and once online using GoToWebinar software. Sessions were promoted over e-mail listservs, on the MHIKNET website, and with physical posters, reaching nearly 2,000 MHIKNET clients.
The objectives of the predatory journal education session were to: 1. Describe MHIKNET's four core library services 2. Describe the history and nature of predatory journals and publishers 3. Demonstrate the problems that arise from the use of these journals 4. Establish how to identify and avoid these journals 5. Learn what fake news is and how to identify it The session was divided into 5 sections addressing each of these objectives. At the beginning of the session participants were polled as to whether they were familiar with predatory journals. In-person session results were tallied by a show of hands, and online session results were tallied using the survey tool within GoToWebinar.
Predatory journals were broadly defined in the session in order to allow participants to understand that these journals rest along a spectrum and that there is not one source or list to easily identify them. Lists of predatory journals (such as Beall's and Cabell's) and comparative lists of "safe" journals (such as the DOAJ, Cabell's, and curated databases) were briefly described, including the difficulty in updating and maintaining them and the potential for error in such lists [44][45][46][47][48]. Participants were instead encouraged to use assessment checklists, such as Rele, Kennedy and Blas's Journal Evaluation Tool [49], and the resources listed on the University of Saskatchewan Library's LibGuide on Assessing a Journal/Publisher [50]. The breadth of these resources is substantial and would be daunting as an entry-point for many health professionals. To provide participants with a starting point in their critical appraisal of scholarly literature, several key points were provided and are summarized in Figure 1. Participants were also encouraged to utilize the CRAP Test (Currency, Reliability, Authority, Purpose) for its simplicity and to emphasize it as a common starting tool in critical appraisal [51]. The section on fake news was included due to its relevance at the time the sessions were being offered and because of the similarity of critical appraisal techniques used for assessing journals and news and media sources. This was not imperative to the predatory journal discussion, but did fit into the education series's larger theme of critical appraisal.

Fig. 1: Key Points to Consider When Critically Appraising Scholarly Articles
Following completion of the education sessions, a voluntary online assessment survey was distributed to session participants; included in Appendix A.
Participants were asked about their awareness of

Communications Practices
• Website competently designed and functional?
• Spelling and grammar errors or distorted images? • Sending out mass emails asking for submissions or links?
• Language that targets authors, rather than readers

Stability of Publisher
• Journal focus on a coherent discipline? • Alarm bells (spelling errors, ludicrous missions) • Journal launch date(s)?
• Are journals empty shells?

Journal Team Members
• Editorial board listed? • Where are the editors located?

Journal Practices
• Information on retraction policies, manuscript handling, and data preservation? • Promises of rapid publication?

Business Model
• Low fee for authors (less than $150 USD)?
• Financial support other than author fees? • Is advertising reputable and relevant? • Does the publisher run (reputable) conferences? predatory journals, what they learned during the session, and their level of confidence in assessing predatory journals. The session has been added to MHIKNET's roster of critical appraisal education sessions, and will be presented again in the future.

Outcomes
A total of 33 participants attended the predatory journal education sessions; 6 in person, 17 by live webinar, and 10 viewed the recording of the webinar. The professions of the participants were diverse, with 8 allied health professionals, 4 nurses, 2 physicians, 5 administrators, 2 analysts, 2 academic librarians, 3 others, and 7 whose professions were unknown. During the sessions participants were asked to express their previous familiarity with predatory journals. Figure 2 shows that their level of awareness of predatory journals was low. Post-session assessment surveys were also conducted to determine the effectiveness of the education sessions. These surveys received 7 responses, 1 (17% response rate) for the in-person session and 6 (35% response rate) for the webinar and its recording. The assessment surveys indicated that while participants began the session with limited or no knowledge about predatory journals, they learned a great deal by the end of the session. Awareness of predatory journals was considerably improved postsession. One hundred percent of respondents (n=7) indicated they learned what a predatory journal is. Figure 3 further describes how much participants learned during the sessions.

Fig. 3: Awareness of Problems Caused by Predatory Journals
When asked in the assessment survey if they would use anything from the session in their work, participants indicated that in the future when they were reading scholarly literature, they would critically appraise materials to assess for predatory sources: "When researching references and articles I will certainly be more aware of some of the pitfalls of predatory journals, and keep a more vigilant eye on these references." "I learned to fully read and make sure what I am reading, is in fact a true study/research" "[I will] Pay more attention to the literature out there, not all journals are created equally" Conversations with participants at the end of the inperson session reiterated that a common outcome of the session was participants' increased awareness of the need to critically appraise all scholarly literature. Overall, the session sparked interest and generated discussion for participants.
Participants showed increased awareness and indicated they would be more attentive to predatory journals going forward, as depicted in Figure 4; though the bulk of participants were only confident they would be able to identify predatory journals some of the time.
In conversation at the end of the in-person session, many individuals expressed hope that MHIKNET librarians would assist them with assessment of journals in the future, and expressed distress that their knowledge of predatory journals had been so limited prior to the session.

Limitations
This paper is a description of an education session offered multiple times in order to address a perceived need. As such, the sample sizes are small and represent a small proportion of health professionals served by MHIKNET. The response rates for the assessment surveys were low, and while participants indicated that they would use the knowledge they obtained in the session, there is no method to assess whether they have done so, nor their effectiveness in doing so. Future research is therefore required to assess the overall state of health professional awareness of predatory journals.

Discussion
Both formal and informal measures indicate that health professionals' awareness of predatory journals is extremely low. It was rare for health professionals to have heard the term predatory journals, and rarer still for them to know what the term meant. As health professionals regularly use and publish research, it is

Fig. 4: Confidence in Ability to Recognize Predatory Journals
critical that their level of awareness of predatory journals be raised. Health professionals need to accurately assess the information they are using, and the journals they are publishing in, with a full grasp of the potential issues of predatory journals. Efforts to raise awareness about predatory journals have largely focused on the academic community and been directed at those researchers looking to publish their work in academic journals. This education session addressed health professionals' awareness of predatory journals, which has not been previously addressed in the literature, despite the degree to which scholarly literature is used in the clinical context. Though the number of participants attending these education sessions was small, the lack of awareness health professionals have regarding predatory journals demonstrates the need for librarians to actively engage in this education.
Education sessions, such as the one conducted by MHIKNET librarians, appear to be an effective mechanism of providing this education. During the sessions participants became familiar with the history of, nature of, and problems associated with, predatory journals. Additionally, session participants were introduced to techniques to assess and identify predatory journals. They indicated limited confidence in their ability to identify predatory journals consistently, but knew librarians could aid them in their assessment. Efforts to raise awareness of predatory journals must therefore be expanded from those who have the potential to publish in predatory journals, to also include those who use journal contents to inform their decisions.

Statement of Competing Interests
No competing interests declared.