Tracking the Impact of Changes to a Provincial Library Service Model: The Results of Two Satisfaction Surveys

Abstract: Introduction: Alberta Health Services (AHS) was created in 2009, merging 12 former health regions and three provincial health authorities. Library services that had previously operated independently across the province were amalgamated into a single provincial entity, Knowledge Resource Service (KRS). A survey of library services was conducted in 2011. Subsequent to that survey, the provincial library service underwent major changes, which culminated in the launch of a provincial library website in August 2013. Another survey was conducted in 2014 to determine the impact of these changes. Methods: AHS staff and physicians were surveyed in 2011 and 2014 using an electronic survey tool. The survey results were analyzed using descriptive statistics and the results reported as percentages. Results: This paper addresses the questions that are comparable between the two surveys. There were 1195 responses to the 2011 survey and 721 to the 2014 survey. Respondents in 2014 had less difficulty accessing the library website. Additionally, more respondents reported that using library resources prevented the occurrence of adverse events, from 13% in 2011 to 36% in 2014. Discussion: The drop in respondents reporting difficulties accessing information via the library website suggests that the new service model is effective in removing access barriers, enhancing the effectiveness of information resources for AHS staff and that the easier to find resources are helping to prevent adverse events.


Introduction
Alberta Health Services (AHS) was created in 2009 by bringing together 12 formerly separate health entities including three geographically based health entities, Alberta Alcohol and Drug Abuse Commission, Alberta Mental Health Board, and the Alberta Cancer Board. Prior to amalgamation, library services, where available, were delivered independently and via both contracted and AHS service providers, resulting in inequities in access to electronic resources and trained professionals between rural and urban areas.
Since the creation of AHS, responsibility for library services has evolved. Libraries are now part of the Knowledge Resource Service (KRS) unit, within the Knowledge Management (KM) department. KM was tasked with optimizing library services including those delivered by contract through the University of Calgary to provide equitable, enterprise-wide access to services and resources to support quality care and evidence-informed decisions.
To support optimization activities, a thorough scan and assessment of existing library services was required. The initial assessment (2011) included a survey of AHS staff and clinician information needs and laid the foundation for a new model to deliver evidence-informed resources and services across AHS [1]. Between 2011 and 2014, the provincial library service underwent significant changes, including the development of a provincial virtual service model that aims to provide equitable access to all AHS staff and healthcare providers, regardless of location. A key aspect of this model was the launch of a provincial library website in August 2013, providing a single point of entry for AHS staff across the province to access resources and services. The website includes topic specific practitioner and patient resources [2]. Additionally, offsite authentication to library resources was implemented in January 2014, allowing staff to access and use library resources from home, mobile devices, or while travelling. A subsequent survey, designed to evaluate the impact of changes, was conducted in 2014. This paper examines the results of these two surveys, asking the question, "Did the improvements in service delivery implemented after the 2011 survey make a difference to AHS staff and clinicians and impact patient care?"

Literature review
The benefits provided by library services and information professionals in clinical environments have been studied for a number of years as all parts of healthcare systems strive to demonstrate their value in the face of competing demands and tightened budgets. Perrier et al. [3] conducted a systematic review of librarian-provided services in healthcare settings that reviewed 25 studies of various methodologies. Perrier et al. [3] found that information professionals, and the resources and services they deliver, save clinicians time, improve patient outcomes, and facilitate the integration of evidence into decision making.
Although assessments of hospital libraries take various methodological forms, surveys are commonly used to identify unmet needs and gaps in service and to seek solutions to meet these needs [4Á7]. The survey itself is also valuable in increasing visibility of the library and its services [4]. In a large multisite study, Marshall et al. [8] used a "critical incident" methodology to survey over 16,000 medical staff about a time when they had sought information for patient care. The authors reported that staff perceived "library and information resources as valuable [and that]. . . information obtained [had] an impact on patient care" (p. 38). Bartlett and Marshall [9] provided a secondary analysis of the Canadian results from the Marshall study, showing that hospital library services positively impact patient care outcomes, including "the avoidance of death", stating that these findings are congruent with other studies. Aitken et al. [10] used a controlled intervention and a pre-and post-survey methodology that found 88% of medical residents changed a treatment plan because of search skills taught by a librarian, and 79% changed a treatment plan based on searches conducted by the librarian. Medernach and Franko [6] assessed library usage and satisfaction after the implementation of new services, which included a comprehensive suite of electronic databases and full-text journals, the addition of two professional librarians, and provision of library training, by surveying a random sample of employees in two questionnaires four years apart. The study found increased use of library services and decreased reliance on general internet searching.
The current study is similar to the one done by Medernach and Franko [6] in that it examines the impact of changes in library resources and services by comparing the results of two surveys conducted three years apart. Their study [6] also replicated some of the questions included in the study by Marshall et al. [8], building on that author's recommendation that further investigation would be important for advocacy and quality improvement. Our study contributes to the literature by demonstrating that improving ease of access to library and information resources increases the use of these resources.

Methods
The two surveys described in this paper were designed and implemented in a similar, although not identical, fashion. Despite these differences, there is value in comparing the results over time to determine the impact of changes implemented after the first survey. Both surveys were distributed province-wide electronically using Select Survey to AHS physicians and staff including nurses, pharmacists, allied health professionals, and researchers. The aim was to ask similar questions to allow for benchmarking and trending across time. Differences in the surveys include different questions, although similar language was used; the method of distribution; and a change from yes/no responses to Likert scales. Results presented here include those questions where sufficient similarities in question design, language, and purpose permit accurate comparisons. The 2011 survey is included as Supplementary Table 1 and the 2014 survey is Supplementary Table 2. 1 The 2011 survey, which is described elsewhere, used a convenience sampling method [1]. The survey was comprised of 18 questions, including some demographic questions, that were developed based on an environmental scan and literature review conducted by the AHS Knowledge Management department. The questions required yes/ no responses. The survey was piloted with individuals recruited by the survey planning team. The pilot group was comprised of front-line, management, and strategy support staff. These reviewers represented both acute and community care, and there was representation from across rural and urban centres. The project team collected feedback and made revisions to the initial draft.
The 2014 survey also used a convenience sampling method. The final 2014 survey was also made available in paper format to reach the broadest audience possible, as many practitioners in AHS do not have regular access to computers in their work environment. Prior to distribution, the survey was pilot tested with 12 representative staff and then refined, similar to the 2011 survey. The 2014 survey questions asked users to rate their experience with library services on a five-point Likert scale from "strongly agree" to "strongly disagree," and included "don't know" and "not applicable" options. The decision to use Likert scales was a methodological improvement to allow for more nuanced answers. For the purposes of this paper, "agree" and "strongly agree" responses were combined and compared with "yes" responses from the 2011 survey. The Likert scale questions were supplemented with qualitative, open-ended questions to enhance clarity and depth of understanding regarding the respondents' experience. Because these open ended questions were only asked in 2014 and cannot be compared with the 2011 survey findings, they were considered out of the scope of this paper but may form the basis of future research.
The final 2014 survey contained 17 questions, and it was made available to all AHS staff and healthcare providers province-wide from 21 April to 5 May 2014. The survey was promoted widely via the KRS website, various zone and discipline-specific AHS electronic newsletters, posters across AHS sites, and direct communication to current users of KRS resources and services.

Analysis
The surveys were analyzed using descriptive statistics, and results were reported as percentages. For questions where responses included the choices "not applicable" or "not aware", or where respondents had skipped a question, valid responses (excluding these response types) were calculated and reported separately.

Results
There were 1195 respondents who provided demographic information to the 2011 survey and 721 to the 2014 survey. AHS employs approximately 96,000 staff. Due to the size of this population, we calculated a sample size assuming a 9 5 confidence interval and a 95% confidence level. The numbers of survey responses in both surveys exceed the calculated sample size of 383.
The ratio of clinical to administrative staff responding to the 2011 and 2014 surveys was similar (Table 1).
AHS is divided into 5 geographic zones: north, Edmonton, central, Calgary, and south. The Edmonton and Calgary zones are the two large metropolitan centres, each with a population of one million people. The remaining three zones together have a total population of one million people.
Overall, the geographic distribution of respondents matches the provincial distribution of AHS staff relatively closely (Table 2), although the north zone is underrepresented in both surveys, and the central zone was highly over-represented in the 2014 survey. In contrast to the 2011 survey, the 2014 survey allowed respondents to indicate that their work was provincial or multi-zone in nature, reflecting AHS's focus on enterprise-wide initiatives.

Use of library
The proportion of AHS staff who reported using the library in the recent past stayed similar between 2011 and 2014, with about three-quarters of respondents reporting that they used library services. Additionally, the most frequently reported barriers to use*lack of awareness and lack of time*remained relatively similar between 2011 and 2014. In contrast, the proportion of AHS staff who reported difficulties accessing information via the library website as a barrier decreased from 17% in 2011 to 4% in 2014 ( Table 3).

Impact of library resources and services
Both surveys probed the impact of library resources and services on organizational priorities such as patient care, research, and more ( Table 4). As questions were not directly comparable, question text from both surveys is included in the table for context.

Discussion
Respondents' views on the impact of library resources and services demonstrate the value of library services in the context of a provincial health authority. Although not all of the questions probing the impact of library services are directly comparable across the two surveys, there were marked increases in many areas. In 2014, 36% of respondents reported that library resources and (or) services helped to prevent an adverse event (AE), this was up from 13% in 2011. Avoidance of AEs is critical in healthcare. The Canadian Adverse Events Study defines an AE as "unintended injuries or complications that result in disability at the time of discharge, a prolonged hospital stay, or death. AEs are caused by health care management (the care provided to patients) rather than the patient's underlying disease process." [11] According to Baker and Norton [11], 37%Á51% of adverse events are preventable. Marshall et al. [12] found 75% of the respondents handled a patient care situation differently as a result of libraryprovided information, thus avoiding an AE [12]. Although our percentage is not as great, the increase indicates that the approach to providing library resources and services is making an impact on the provision of patient care. This echoes Bartlett and Marshall's [9] analysis that showed information resources and services provided by the library support positive patient outcomes. In addition to avoiding adverse events, there was a marked increase (from 42% to 90%) in respondents who agreed that library resources and (or) services saved them time. These increases may reflect the increased ease of accessing all information from a single source. Finally, 71% of 2014 respondents indicated that information was used for patient or family education, up from 38% in 2011. The provincial library website launched in 2013 contains specific sections dedicated to general and topic-specific patient resources, which were not previously available province-wide.
The drop in respondents reporting difficulties accessing information via the library website between 2011 and 2014 (from 17% to 4%) suggests that the new website with these topic-specific pages is effective in its aim to remove barriers to accessing information resources. Additionally, the number of respondents indicating that they access library services elsewhere dropped from 18% in 2014 to 5% in 2011, reinforcing the idea that the new service model is meeting AHS staff and clinician needs. Staff that provide direct patient care often cite a lack of time and searching skills as barriers to accessing evidence-based information.
We were surprised that the number of respondents citing lack of awareness of library services increased by 11% between 2011 and 2014. It is possible that due to the change in the name and associated branding of the library service between 2011 and 2014, respondents are still learning that KRS represents library services. This highlights the ongoing need for increased promotion and outreach.
Our research demonstrates that investments in improving and streamlining service delivery models make it easier   for healthcare professionals to access evidence resources, and that these resources are increasingly being used to provide efficient and effective patient care. There is value in deploying similar survey instruments as library services adapt to meet the needs of their user base: the resulting evidence helps the library system evaluate the impact of changes.

Limitations
The size of the changes reported here may not be exact due to the differing nature in the number of questions and the way the questions were designed, formatted, and asked. These factors can affect responses. Additionally, the overrepresentation of central zone respondents in 2014 may have skewed the results due to the many rural sites that may have benefitted more from the new provincial service model than their urban counterparts located in other geographic locations.
These two surveys gathered the feedback of a relatively large number of staff and healthcare providers, working in a wide range of roles (including clinical, policy development, and leadership functions) in a provincial health authority. Because both surveys relied heavily on convenience sampling, it is likely that the respondents are not representative of AHS staff and healthcare providers in general, but rather over-represent current users of library resources and services [6]. However, by surveying the same population both before and after the introduction of major service delivery changes, these two surveys offer a unique glimpse into how changes in library services are received by a large healthcare workforce. It would have been valuable to ask participants if they had responded to the previous survey and what changes they found in the library. As Medernach and Franko [6] note, "[m]easuring impacts at one point in time does not create an accurate picture of the impact of the services and information that the library provides". Maintaining similar questions in both surveys allows the library service to establish benchmarks and monitor changes over time.
The number of respondents reporting that the library helped them avoid an AE was a positive result in our study. However, the term was not defined in the survey, so responses may reflect a variety of interpretations of an AE. It would be interesting to follow up with a selection of clinicians, perhaps using a more qualitative methodology, to probe more deeply into this finding.

Conclusion
The changes implemented between the 2011 and the 2014 survey, including the new province-wide website and remote authentication, have improved AHS practitioners' access to AHS licensed information resources and decreased practitioner reliance on other tools. Between 2011 and 2013 respondents reporting that library resources helped to prevent an AE changed from 13% to 36%, attributable to the increased ease of use and discoverability of these resources.
The provincial library service for AHS continues to undergo change. In late 2014, AHS announced that it would be renegotiating the contract with the University of Calgary, initially bringing all contracted library staff directly into the AHS organization. Additionally, the library service is dedicating resources to license, promote, and provide education around a number of point of care tools aimed at specific clinical groups, including physicians, nurses, and rehabilitation professionals. It is anticipated that another library satisfaction survey will be implemented in 2016, providing yet further opportunity to measure how resources and services are perceived by staff, as both the library service and the delivery of healthcare continue to evolve.