The Value of Library and Information Services in Patient Care: Canadian Results From an International Multisite Study

Introduction: This paper presents the Canadian results from a larger, international study with the objective of assessing the value of health library and information services and their impact on patient care. Methods: Data were collected using a web-based survey of healthcare providers in 13 Canadian hospitals collectively served by four libraries, and data were analyzed statistically using SPSS. The survey centred on a specific, recent, patient care incident for which the respondent had sought information. Follow-up semi-structured phone interviews with librarians at the participating sites provided supplemental data. Results: Twelve hundred and thirty-one people from the Canadian sites responded to the survey. Over 70% indicated that their management of the clinical situation changed as a result of the information. Positive changes included advice given to patient or family (48%), choice of drugs (31%), and choice of treatment (30%); adverse events that were avoided included patient misunderstanding of disease (23%), additional tests or procedures (18%), and patient mortality (5%). Results also showed which information resources were used and from where they were accessed. The information resources were valued as much, if not more, than other sources of information such as laboratory reports or medical records. Discussion: The results showed that participants perceive health library and information services to be highly valued and reported that their use has a positive impact on a range of patient care outcomes. They also highlighted the preferred information resources and access points among different groups of health professionals.


Introduction
Over the past few decades, a number of studies have investigated the value and impact of libraries and library services in a healthcare context. Although this research has been international in scope, including work from Australia, the United Kingdom, and the United States, there has been little work done in Canada.
In 2007, the National Network of Libraries of Medicine, Middle Atlantic Region (NN/LM MAR) launched a multisite study into the value and impact of health library and information services on patient care outcomes. This paper presents a secondary analysis of data from the four Canadian libraries that participated in that study.

Background
This research builds on a history of research into the impact of libraries, information services, and information resources on patient care. In 1986, King studied physicians, nurses, and other health professionals who had requested information from their hospital library and asked what impact the information would or might have [1]. Almost two-thirds of the participants indicated that they would definitely or probably manage their cases differently based on the information provided by the library. This study, however, did not follow the participants to determine if, in fact, their practice did change.
In 1992, Marshall conducted the Rochester study in response to a need for evidence of the impact of hospital library and information services on patient care outcomes [2]. This landmark study explicitly studied this relationship by asking physicians to request information related to a specific clinical case and to evaluate the impact of that information on patient care. Eighty percent of the physicians indicated that they definitely or probably managed an aspect of patient care differently. Changes in care included: choice of tests (51%), choice of drugs (45%), and diagnosis (29%). The physicians also indicated the adverse events avoided as a result of the information including: patient mortality (19%), surgery (21%), and additional tests or procedures (49%). Finally, the physicians rated the information provided by the library more highly than that from other sources such as lab tests, diagnostic imaging, or discussion with colleagues. Since its publication, this work has been a benchmark for demonstrating the value of library and information services in supporting patient care and their positive impact on clinical outcomes. It has been highly cited within both library and information science and medical literature.
Pluye and colleagues investigated the reasons for which physicians sought information from electronic information resources [3]. They identified seven reasons: answering clinical questions or clinical decision-making, fulfilling educational objectives, satisfying curiosity, overcoming memory limitations, sharing information with patients, exchanging information with other health professionals, and managing tasks with other health professionals. This group later studied residents, nurses, and pharmacists, assessing their use of a specific e-resource, which they were instructed to search on a regular basis to find information in support of treatment recommendations [4]. Their findings indicated that in 35% of the situations they described, the information use was associated with positive patient outcomes. These included increased patient knowledge, avoidance of unnecessary interventions, prevention of disease or morbidity, health improvement, and increased patient satisfaction.
In a 2012 survey of Australian health professionals, 83% of respondents indicated that library and information services helped them improve health outcomes for their patients, whereas 76% indicated that the information they received improved their diagnosis or treatment plan [5]. However, the study methodology did not delve into the particulars of the specific health outcomes or changes to diagnosis and treatment plans.
In recent years, there have been a number of smaller scale studies into the use of health information services, including a study of four hospitals in Colorado and Missouri [6] and another focused on American Veteran's Administration hospitals [7].
Other research has focused on specific types of library services such as clinical librarians [8], ''just-in-time'' services [9,10], or a particular application of information to support decision-making [11]. Later work by Urquhart and her group was foundational to the research reported in this paper [12].
Beginning in 2007, Marshall and her research group at the University of North Carolina at Chapel Hill revisited the Rochester study on behalf of the National Network of Libraries of Medicine, Middle Atlantic Region (NN/LM-MAR) [13,14]. In addition to updating the original Rochester study methodology, this new research included a much larger sample. Over 16 000 physicians and nurses from 118 hospitals responded to the survey. The methodology followed a critical incident technique approach, whereby participants responded in the context of a specific clinical case for which they had spontaneously sought and used information. Thus, the findings represent actual clinical practice, without the decision to seek information being influenced by the study methods. In spite of the dramatic changes to the information landscape in the intervening two decades, the findings again demonstrated the value and impact of information services on patient care outcomes. Overall, three-quarters of the participants indicated that they definitely or probably managed the clinical case differently. Advice given to a patient or family was the most common change (48%), followed by choice of drugs (33%), choice of treatment (31%), diagnosis (25%), and choice of test (23%). Key adverse events avoided as a result of the information included: additional tests or procedures (19%), misdiagnosis (13%), adverse drug reaction or interaction (13%), and patient mortality (6%). Participants again rated the information provided by the library as more important than that from discussion with colleagues, lab tests, and diagnostic imaging.

Methods
This paper presents a secondary analysis of data from Canadian sites from the NN/LM MAR study, with a datasharing agreement allowing the Canadian data to be shared between the UNC research team and McGill University. The original mixed-methods study included focus groups of librarians for planning purposes; a webbased survey of physicians, residents, nurses and nurse practitioners; and a series of follow-up interviews with selected survey respondents [14]. This paper only reports findings from the Canadian participants of the survey (1231 of the 16 122 survey participants were from Canadian sites). This paper also reports on the findings from new data collected through follow-up interviews with librarians from the Canadian sites.

Survey
The first part of the survey asked respondents to identify both their profession (physician, resident, nurse, nurse practitioner, or other) and the type of work their job involved (patient care, management/administration, clinical research, education, or other). Although respondents could select as many work roles as were applicable, only those who selected ''patient care'' or ''clinical research'' were able to proceed with the survey. Because the survey was centred around the use of information for clinical care, this requirement was to ensure that participants were more likely to be able to respond to the survey questions. Following a critical incident technique approach [15,16], the survey then prompted respondents to think ''of an occasion in the last 6 months when you looked for information resources for patient care (beyond what is available in the patient record, EMR system, or lab results)''; the remaining questions in the survey were to be answered in the context of this patient case.
Respondents next selected the best description of the primary diagnosis of the patient (from a list of 19 options such as cancer or heart disease) and the type of information (e.g., drug information, therapy information, clinical guidelines) that was needed to answer the question. A set of questions asked respondents about where they obtained the information. This included a list of key health sciences information resources (e.g., Medline, CINAHL, Dynamed, STAT!Ref) generated in consultation with the Value Study Planning Group; not all sites necessarily had access to all resources, and sites might also have had resources not included in the list. Respondents specified which resource(s) they used, how they accessed the resource (e.g., from an institutional website, personal subscription, asked a librarian), and from where they conducted the information search (e.g., office, library, patient care unit), as well as whether they found the information they needed and which resource(s) that had been searched contained relevant information. A final set of questions asked about the value and impact of the information. This included whether the clinical situation was handled differently as a result of the information, an assessment of the value of the information (e.g., relevance, clinical value, contribution to higher quality care), whether the use of the information led to positive changes in patient outcomes (e.g., reduced length of stay, choice of treatment) or avoided adverse events (e.g., adverse drug reaction, misdiagnosis, mortality), and how valuable the information was considered in reference to other sources such as diagnostic imaging or lab tests. The full survey is available as an appendix to Marshall's 2013 paper [14].

Recruitment and sample
Following initial recruitment of sites from within NN/ LM MAR, the study was opened to other sites across Canada and the United States. The study team provided each site with a Facilitator Handbook [14] that detailed the steps in the study and the respective roles of the facilitator (typically the hospital librarian) and the research team. One task of the facilitator was to identify one or more ''champions'' within the organization who would support and endorse the study within the organization and co-sign the e-mail invitation to participate in the study. Another was to obtain research ethics approval within their own institution, if required. Ethics approval for the multisite study was obtained from the University of North Carolina Institutional Review Board (UNC IRB). The UNC team provided support to the facilitators in the ethics review process as necessary.
The UNC research team managed the registration of sites and provided support to facilitators of all sites that expressed interest in the study. The study planning group members also served as mentors to the site facilitators.
From the library sites that initially expressed interest, four Canadian sites met all the requirements for participation. A number of additional Canadian sites were originally interested but were unable to participate for a variety of reasons. Anecdotal reports suggested that one reason was provincial legislation, which specified that data from healthcare sites could not be stored on servers outside of Canada. Another was the absence of approval from the institution's internal ethics review board.
Following a pilot survey in fall 2010, the full survey was implemented in spring 2011. All physicians, residents, nurses, and nurse practitioners within each participating organization were invited to participate in ''a study on the value of information in clinical settings'' and advised that their ''answers to the survey questions may ultimately help our institution provide better patient care by supporting you with the information you need, when and where you need it'' [14].

Follow-up interviews
In the summer of 2013, follow-up interviews were conducted with librarians who facilitated the original survey within their respective hospitals. The semistructured interview guide focused on two main themes, the librarian's experience as a participant in the study and the impact on their library and practice of the study and its findings. Interviews were conducted by phone and were audio-recorded and later transcribed. The transcripts were analyzed with open coding, to capture the themes that emerged from the perspective of the respondents [17]. Two of the four Canadian librarians who were involved in the study participated in the interviews.
Ethics approval for the interviews, and the secondary analysis of the survey data from the Canadian sites was obtained from the McGill University Ethics Review Board.

Results
The four Canadian libraries served 13 hospitals in three provinces. Seven hospitals had between 101 and 299 beds, four had between 300 and 499 beds, and two had 500 or more beds. All 13 hospitals were located in urban or suburban areas; some hospitals included a teaching role in their function, although they were not all officially designated as teaching hospitals.
Twelve hundred and thirty-one healthcare providers from the various hospitals responded to the survey. As not every respondent answered every question, the total number of responses to some questions may be below 1231. More women (576, 46.8%) than men (336, 27.3%) participated, due in part to the large number of nurses. Table 1 shows nurses and physicians were the largest groups of professionals followed by medical residents and nurse practitioners. Their work involved a range of activities (Table 2). Overall, most respondents were from two broad age ranges; 397 (32%) were aged 25Á44 and 459 (37%) were aged 45Á64. Almost half the respondents (49.2%) had been in practice for over 20 years, with the next largest proportions having been in practice for 2Á5 years (13.8%) and 6Á10 years (12.6%).

Information resources used
As specified in the survey, all responses were in the context of a single, specific patient care incident. Within that context, respondents listed all of the information resources they used in response to the clinical information need (Table 3). Overall, the top five were e-Journals (46%), PubMed (45%), e-Books (32%), UpToDate (31%), and e-Medicine (24%). It is interesting to note the selection of resources was not consistent among the different groups of professionals. UpToDate was highly used by residents (78%) but not as much by the others. By contrast, CINAHL was more highly used by nurses (14%) and nurse practitioners (37%). For each resource used, respondents indicated how they accessed the resource. Aggregated results are shown in Table 4. Overall, the institution's library website or intranet were the most highly used (46% and 45%, respectively), closely followed by general Internet search engines (40%). Although there was a preference for electronic resources, the institution's physical library was used by almost onefifth of the respondents. The librarian was also an access point for information (17%) and was consulted more frequently than colleagues (12%).
Respondents also indicated where they were physically located when they conducted the search for information (Table 5). Overall, the patient care unit (50%) and the office (46%) were the most common locations, though the most commonly cited location varied by profession. Physicians most often searched from their office (71%), whereas a similar proportion of nurses searched from the patient care unit (70%). Interestingly, although the library was only listed by 13% of respondents overall, over 30% of residents indicated that they conducted or requested the search from the library.

Value and impact of information
Respondents assessed the information by indicating whether or not they agreed with a number of statements regarding the information ( Table 6). The statements addressed four areas: quality of the information (e.g., the information was relevant), its cognitive value (e.g., the information provided new knowledge), its contribution to quality patient care (e.g., the information was of clinical value), and time saved. Assessment of the information was overwhelmingly positive; overall, agreement ranged from 78% to 100%. In most cases, 90% or more of the respondents agreed with the statements. The lowest level of agreement was with the statement that the information saved time, with an overall score of 78%.
In terms of the impact of the information, respondents were asked whether or not they handled the clinical situation differently as a result of the information (Table 7). Overall, over 70% of respondents indicated that they definitely or probably handled the situation differently. Results from physicians, residents, and nurse practitioners were relatively      consistent. For nurses, the responses were mostly ''probably yes'' (42%) or ''probably no'' (35%). The impact of the information was further detailed with questions regarding specific patient care outcomes. Table 8 shows the responses to the question of whether any of a set of outcomes changed in a positive way as a result of the information. Overall, ''advice given to patient or family'' was the most cited outcome (48%); an even greater percentage of nurse practitioners (69%) indicated this change. Overall ''choice of drugs'' and ''choice of treatment'' were indicated by 31% and 30% of the respondents, respectively.
In addition to documenting positive changes to patient care outcomes, respondents also indicated whether any negative or adverse events were avoided as a result of the information (Table 9). ''Patient misunderstanding of disease'' was the most cited adverse event that was avoided (22%). As with the ''advice given to patient or family'', this response was also most indicated by nurse practitioners (40%). Other adverse events that were avoided included ''additional tests or procedures'' (18%), ''adverse drug reaction or interaction'' (12%), and ''medication error'' (9%). Perhaps most significant from a human standpoint, 49 respondents (5%) indicated that ''patient mortality was avoided as a result of the information''.
Finally, respondents evaluated the information resources provided by the library along with other sources of information that are also involved in patient care. Table 10 shows the number of respondents who rated an information source as either ''important'' or ''very important'' on a four-point scale. Overall, the information resources that respondents had used (listed in Table 3) were rated highly by more respondents (96%) than the other three categories of ''discussion with colleagues'' (90.6%), ''laboratory tests'' (82.9%), and ''diagnostic imaging'' (77.1%).

Follow-up interviews
Librarians whose libraries and institutions participated in the study received a summary of the study findings in PowerPoint format, as well as the dataset of survey results that they could further analyze. One focus of the interviews was to understand whether the individualized study results were of benefit. Librarians from two of the four Canadian sites participated in the interviews.
Both librarians found their involvement in the study to have been a positive experience, with potential or actual positive outcomes for the library. One benefit was the identification of the individual resources that healthcare providers served by the library chose to search in response to a clinical information need. Rather than only considering aggregate results, it was considered valuable to make an explicit link between the use of library-based resources and the impact on patient care outcomes within the institution. The results of the survey were considered useful and valuable to the librarians in advocating for the library and library services within the larger institution and in demonstrating the value of the library. This benefit was manifested in different ways depending on the circumstances of the individual libraries. In one case, the library currently enjoyed strong institutional support; therefore, there was not an immediate need to further advocate for the library. However, the librarian stressed that this type of support could not be taken for granted and that having data explicitly showing the link between the library and its services and positive patient care outcomes would be essential to ensuring the library's ongoing support and continued success.
In the other case, the library and broader institution were involved in changes, and the study findings had a much more concrete impact. The librarian was able to use the results of the study to demonstrate the value and importance of the library to new people, particularly those in senior administration, who didn't already know about the library or value its services. The study findings were also successfully used to support the replacement and upgrading of library staff.
The librarians also discussed some of the challenges they encountered during the survey and their suggestions for future research. One challenge was the absence in some hospitals of a comprehensive e-mail distribution list. This meant that some staff might have missed receiving a direct invitation to participate in the research. One alternative approach was to post notices in common areas, such as at the nursing station, informing staff about the research and providing a link to the web survey. These challenges in reaching staff may have had a negative impact on the response rate.
''. . . challenge is that not every nurse . . . actually has an e-mail so reaching the nurses was the hardest group and we had to put a link on our intranet page and direct people to that link because there's no group e-mail to e-mail all of the nurses.''(Librarian 1) Another challenge mentioned was in obtaining an accurate count of the number of staff. In some multisite institutions, an individual may work at more than one site. Depending on how the institution manages its staffing records, this could lead to staff being double counted, thus artificially lowering the measure of the response rate.
One suggestion for future research was to expand the scope to include those who, while not directly providing patient care, do have an influence on patient care outcomes. The example was given of nurse managers who might still search for and use information to affect patient care through activities such as establishing departmental policy.
''We have five that are master level prepared nurses that do nothing but work on patient flow. They've actually assigned people managerial roles and taken them off the floors so that they can actually make the length of stay shorter and at the same time make it a good patient outcome, because your floor nurse hasn't got time. (Librarian 2) Another comment was that the original survey did not necessarily include all of the specific resources held by the library. It was suggested that future surveys might include a list of resources customized to match the holdings of individual participating libraries.

Discussion
Overall, the results of this study showed that hospital library and information services have a positive impact on patient care outcomes, with outcomes of their use including the avoidance of death. The findings from the Canadian sites were consistent with those of the full multisite study. However, of more interest is the fact that these recent findings were not inconsistent with those of other studies, including for example, the Rochester study of 1992. So, although the information landscape has changed remarkably over the past two decades, the value of health libraries and their services remains strong.
In spite of the increase in the number and range of specialized information resources, three of the top five most used resources (e-journals, e-books, and MEDLINE) were quite conventional, albeit in digital format. Even print books and journals were among the top seven. One reason for this may be that these resources are ubiquitous; other resources might not have been held by all participating libraries. However, the results also suggested that traditional resources are still valuable and useful. Likewise, the library or the institution's websites were the most frequently used access points for the information, again reinforcing the value of the library. Although the physical library was used less that 20% of the time, this does not suggest that the library is declining in value. Rather, it was shown that the library continues to reach and serve the users through digital means. It is also noteworthy that the librarian was specifically cited as a source of information and more highly ranked than colleagues.
The results also showed variations among the different groups of health professionals, perhaps reflecting variations in the scope of their practice and their responsibilities. The results from the nurse practitioners sometimes were similar to physicians or residents (e.g., managing the clinical situation differently, outcomes such as choice of drug or treatment), whereas in other cases results were more similar to nurses (e.g., accessing resources via the intranet). Variation was also seen for the physical location from which the search was conducted or requested and the resources used. These findings could be applied to targeting resources and services to ensure that healthcare providers are able to access the information they need, where they need it. For example, because nurses most often searched or requested information from the nursing station using the intranet, libraries should ensure that resources are accessible in this way.
This study relied on the respondents' recall of a particular patient care situation and their management of it. Although the critical incident technique does mitigate the problem of recall bias, it is still possible that respondents would have selected an incident for which the use of information affected their management of the clinical situation. As such, it is unclear to what extent these results can be generalized to all clinical situations. At the same time, those who did not have a successful search or information use experience may also have chosen to respond to the survey to document an unsatisfactory situation.
These results showed a small snapshot of the value and impact of library and information services on patient care outcomes in Canadian hospitals. However, these four libraries with the 13 hospitals they serve are not representative of the full Canadian healthcare system. Additionally, American findings are not necessarily generalizable to the Canadian context. For this reason, we plan a follow-up study of a broader sample of Canadian healthcare libraries and institutions; by following a similar study design to the NN/LM MAR study, the goal is to recruit a broader sample from across Canada. The sample should reflect the linguistic, geographic, and health system difference across Canada, including among other factors: urbanÁsuburbanÁ rural settings, provincialÁterritorial health systems, and specialist versus general settings.

Conclusions
This research showed that within the participating hospitals, library and information services were highly valued, and their use had a positive impact on a range of patient care outcomes. Understanding the information resources used, and the preferred means of access, will allow libraries to deliver services and resources to address the varied needs of different segments of their user populations. Ultimately, by ensuring health professionals have access to the information they need when and where they need it, positive patient care outcomes will be supported.