RESEARCH ARTICLE / ARTICLE DE RECHERCHE
Ashley Farrell and Jeff Mason
Ashley Farrell.2 Prevention & Cancer Control, Cancer Care Ontario, 505 University Avenue, 18th Floor, Toronto, ON M5G 1X7.
Jeff Mason. Health Sciences Library & Archives, 0B-Regina General Hospital, 1440-14th Avenue Regina, SK S4P 0W5.
1This article has been peer reviewed.
2Corresponding author (e-mail: ashley.farrell@cancercare.on.ca).
JCHLA / JABSC 35: 116–123 (2014) doi: 10.5596/c14-030
Abstract: Objectives: To evaluate the impact of literature searching services on patient care, and to create a validated quick-assessment tool to be used by other libraries to assess their own literature searching services. Methods: All users of the Regina Qu'Appelle Health Region Health Sciences Library who requested a literature search for the purposes of patient care were emailed a link to a short survey as a preamble to search results sent using LibAnswers. A reminder was sent one week after the initial invitation. Responses were collected using FluidSurveys. Face and content validation of the survey were conducted with prospective respondents, librarians, and research support staff followed by a short pilot phase to assess reliability. Results: Fifty-four responses were received for a response rate of 57.5%. Immediate impacts of the information provided included confirming, changing, or determining a diagnosis (7.1%) or treatment plan (64.3%); avoiding adverse events (9.5%); and preventing (4.8%) or initiating (2.4%) a referral or consultation to another department. Future uses for the information provided include changing the approach to particular (27.8%) or future (55.6%) patients, sharing with colleagues (68.5%), and teaching (42.6%). Conclusions: Libraries do effect change in patient care. It is possible for hospital libraries to assess the impact a service such as literature searching has on patient care without requiring a major time investment from library users. Librarians in similar settings are encouraged to further validate and use this tool to more easily compare the impact hospital libraries have on patients.
The Regina Qu'Appelle Health Region (RQHR) is an integrated health system located in and around Regina, Saskatchewan, Canada. The region provides hospital, rehabilitation, community and public health, long-term care, and home care services to over 260,000 people living in cities, towns, villages, rural municipalities, and Indigenous communities in the southern half of Saskatchewan.
From 1 April 2013 to 31 March 2014 the RQHR Health Sciences Library (HSL), a medium-sized hospital library system, answered over 800 literature search requests from library users. The self-selected primary purpose of 36% of these questions was patient care.
In 2013 the HSL held a strategic planning retreat to identify measurements of success for the library. One outcome of this retreat was the realization that the measurements currently collected about the library's services do not clearly demonstrate the library's role in patient care. As a result, the library decided to investigate how one of its services, literature search requests, might have “a direct effect on patient care” [1].
Issues regarding the measurement of success of library services are not unique to the RQHR library. Most hospital libraries collect various usage statistics to describe and document the services they offer to their users. The utility of these measures is disputed and a need for hospital librarians to determine how their work impacts outcomes such as patient care has been identified [2, 3].
A wealth of literature exists that describes how libraries can measure the impact their services have on patient care outcomes [1–18]. Most library impact studies employ the “critical incident technique” asking respondents to think back on an incident where they used information the library sent them for their work. This technique has been used and validated in hundreds of studies [4, 19]. However, as described by Bryant et al. [19], asking respondents to think back on a specific event can create problems such as memory bias or “elicit[ation of] an ‘over-rosy’ description of events to please the researcher”.
Many library impact studies look at library services broadly, encompassing offerings such as user-led searches, librarian-mediated searches, interlibrary loans, information literacy training, and clinical librarian services [4]. Fewer studies look specifically at literature searching. Two recent poster presentations at the Canadian Health Libraries Association/Association des bibliothèques de la santé du Canada 2014 conference examined librarian-mediated searching. Hurrell et al. [20] examined the usefulness and effectiveness of a literature searching service but did not focus exclusively on patient care questions. Konrad et al. [21] examined strengthening the validity of a library's literature search assessment survey. The Quality Development Team report on a three-month study where a print survey was attached directly to search results for all literature search requests [4, 7]. Although the previous studies examined literature searching services for all types of questions, both Johnson [5] and Klein et al. [6] examined the impact of mediated literature searches on patient care outcomes. In the former, a 10-question retrospective survey was mailed out to mental health professionals who had requested a search in the previous eight months [5]. In the latter, patient records were prospectively examined to compare the length of stay and patient costs when a literature search was requested with a control group [6].
One noted limitation of impact research is that no single set of validated questions exists for assessing the impact of library services, making it difficult to compare studies directly [1, 3, 4]. Weightman et al. [1] began developing a set of recommended impact questions that were piloted but concluded further validation was required. In the case of studies focused specifically on literature searching services it is also difficult to make conclusions about impact due to confounding factors and recall bias due to the study design [4]. In addition, the complexity of the study design and time required for Klein et al. [4] to conduct their study were identified as possible limitations on the ability of other researchers to reproduce results.
To improve future research, best practice guidance on how to conduct and design an impact survey has been developed [1]. Key recommendations include enhancing response rate by offering an incentive, keeping the questionnaire as brief as possible, and following up with two to three reminders [1, 2]. Other recommendations to improve impact surveys are: using a web-based questionnaire to save time during administration and data analysis of the survey and minimizing bias through the use of researchers independent from the study [1]. Bryant et al. [19] also suggested, as an alternative to the critical incident technique, use of a brief questionnaire sent at the same time as the response to reduce prestige bias to which library use studies are prone.
Because of the limitations of previous research and the need for a set of validated questions, this study set out to develop a validated survey built upon previous assessment tools such as Marshall's 2013 update to her values study [12] and Weightman's best practice guidelines [1]. The aim of the survey was to determine quickly (i.e., under five minutes) how literature searches conducted by hospital librarians impact patient care.
All library users who indicated the primary purpose of their question was patient care were invited to participate in a short survey. Users who asked more than one patient care question during the time the survey was running were sent an invitation to participate for each question asked. If more than one library user requested a response to the same question, separate invitations were sent to each person. All responses were collected anonymously.
Survey invitations were included as a preamble to the answers sent by a librarian through LibAnswers (Springshare, Miami, FL) in response to each question. A reminder to participate was sent one week after the initial invitation. Upon completion of the survey, respondents were directed to a second separate survey where they could sign up to receive a five dollar coffee card that was offered as a small incentive to encourage participation in the survey. Reminders and coffee cards (along with a thank you note) were sent by a library technician to maintain anonymity of respondents.
The survey was designed using FluidSurveys (Fluidware, Ottawa, ON) with input and support from staff in the health region's research department. Branching and survey logic were used to minimize the number of questions presented to respondents (Figure 1). Depending on the answers selected, respondents were presented with 4 to 6 questions. All questions were multiple choice with an open-ended option when applicable (see Appendix A for a copy of the survey questions).
Fig. 1. Branching and survey logic were used to reduce the number of questions asked; 4–6 questions were asked depending on how the respondents answered the questions.
Librarians, research staff, and prospective respondents evaluated the survey to assess face and content validity. These reviewers were asked to determine, based on a brief description of the survey's purpose, whether or not the survey appeared to measure what it purported to measure (i.e., impact of literature searching services on patient care). In addition, reviewers were asked to examine the language (content) used in the survey to ensure respondents would be clear what they were being asked. This review process resulted in minor changes to the language used in the survey and were incorporated prior to its use.
Survey results were analyzed using FluidSurvey's built-in reporting capabilities.
Based on the library's past year of patient care literature search requests, a total of 50 responses were desired to ensure a sufficient sample size.
Ethics approval for the survey was obtained from the Research Ethics Board of the Regina Qu'Appelle Health Region.
A three-week pilot began on 27 January 2014 and was completed without issue. All pilot responses were included in our analysis. The first invitation was sent on 28 January 2014. The last invitation was sent on 12 June 2014 and the survey closed on 26 June 2014.
Ninety-four invitations were sent and 54 responses were received for a response rate of 57.5%. The median time to complete the survey was 1:46 min (Interquartile range 1:07–2:18). Of the 54 responses, the five most frequent responding groups were: physicians (18, 33.3%), nurses (12, 22.2%), therapists (9, 16.7%), pharmacists (6, 11.1%), and residents (4, 7.4%) (Table 1).
| Response | No. (%) |
|---|---|
| Physician | 18 (33.33) |
| Nurse | 12 (22.22) |
| Pharmacist | 6 (11.11) |
| Resident | 4 (7.41) |
| Dietitian | 1 (1.85) |
| Therapist* | 9 (16.67) |
| Manager/Administrator | 1 (1.85) |
| Other Staff | 3 (5.56) |
| Total | 54 (100) |
| *Therapist includes physiotherapist, occupational therapist, speech language pathologist, and respiratory therapist. | |
Respondents were asked to indicate whether or not the HSL had answered their question. The majority of respondents (42, 77.8%) indicated their question was answered (Table 2). Ten (18.5%) respondents stated their questions were only partially answered citing reasons such as the answers led to more questions (2, 20%) or key points of the question were not answered (1, 10%). Other responses (9, 90%) included: “didn't provide direction for a decision”, “it was an obscure topic so there was not much research on the topic”, “information was not reported in the literature”, and a need to refine the search request. Two (3.7%) respondents said that the information did not answer their question for reasons such as insufficient detail in the question asked or because no answer appeared to exist.
| Response | No. (%) |
|---|---|
| Yes | 42 (77.8) |
| No | 2 (3.7) |
| Partially | 10 (18.5) |
| Total | 54 (100) |
The 42 respondents who indicated the library had answered their question were asked about the immediate impact of the information on patient care (Table 3). Most respondents (27, 64.3%) stated that the answer had an immediate impact on the treatment or management of a patient. Less frequent uses of the information included refreshing memory (8, 19.0%), avoiding an adverse event or critical incident (4, 9.5%), diagnosing a patient (3, 7.1%), and preventing a referral or consultation (2, 4.8%). Other immediate impacts respondents identified included: “[for a] guide in utilizing monitoring equipment”, “roles and scope of practice of nurses”, “enhancing the [patient's] knowledge of their condition”, and “provided information for … a new procedure”. Seven (16.6%) respondents indicated the information would not immediately impact patient care.
| Response | No. (%) |
|---|---|
| There was no immediate impact | 7 (16.7) |
| Diagnosis (confirmed/changed diagnosis and/or diagnostic test or procedure) | 3 (7.1) |
| Treatment/management (confirmed/changed drugs, treatment, and/or post-hospital care plan) | 27 (64.3) |
| Refreshed memory | 8 (19.0) |
| Avoided adverse event(s)/critical incident(s) | 4 (9.5) |
| Prevented referral to or consultation by another department | 2 (4.8) |
| Initiated referral to or consultation by another department | 1 (2.4) |
| Other impacts not listed | 13 (3.1) |
| Total unique responses* | 42 |
| *Respondents could select all that applied; therefore, the column does not equal 100% | |
Respondents who said the information provided by the HSL had an immediate impact on the diagnosis, treatment, or management of a patient were asked to answer a follow-up question describing the impact in more detail. Of the 27 respondents who reported an immediate impact on the treatment or management of a patient, six (22.2%) said the information provided by the HSL determined their choice of drug(s), eight (29.6%) said the information confirmed their proposed choice of drug(s) used, and five (18.5%) said the information changed the choice of drug(s) used (Table 4). Asked about other types of treatment or management excluding medications, eight (29.6%) said the information provided determined proposed treatment(s), seven (25.9%) said the information confirmed proposed treatment(s), and three (11.1%) said the information changed other specific treatment(s). Other responses about the immediate impact of the information provided by the HSL on the treatment or management of patients included: “made sure appropriate [rehab] goals were addressed” and “obtained coverage of [a] drug for [a] patient”. Only three respondents (7.1%) reported an impact on the diagnosis of a patient and selected responses ranged from changing a diagnosis to changing the choice of diagnostic tests ordered (Table 5).
| Response | No. (%) |
|---|---|
| Determined choice of drug(s) used | 6 (22.2) |
| Confirmed proposed choice of drug(s) used | 8 (29.6) |
| Changed proposed choice of drug(s) used | 5 (18.5) |
| Determined other proposed treatment(s) | 8 (29.6) |
| Confirmed other proposed treatment(s) | 7 (25.9) |
| Changed other specific treatment(s) | 3 (11.1) |
| Determined post-hospital care plan | 2 (7.4) |
| Confirmed proposed post-hospital care plan | 1 (3.7) |
| Changed proposed post-hospital care plan | 0 (0) |
| Other | 3 (11.1) |
| Total unique responses* | 27 |
| *Respondents could select all that applied; therefore, the column does not equal 100%. | |
| Response | No. (%) |
|---|---|
| Confirmed a diagnosis | 1 (33.3) |
| Changed a diagnosis | 1 (33.3) |
| Confirmed the choice of diagnostic test(s)/procedure(s) (e.g., CT, blood work) | 2 (66.7) |
| Changed the choice of diagnostic test(s)/procedure(s) (e.g., CT, blood work) | 1 (33.3) |
| Ordered additional diagnostic test(s)/procedure(s) (e.g., CT, blood work) | 1 (33.3) |
| Avoided ordering of additional/unnecessary diagnostic test(s)/procedure(s) (e.g., CT, blood work) | 1 (33.3) |
| Other | 0 (0) |
| Total unique responses* | 3 |
| *Respondents could select all that applied; therefore, the column does not equal 100%. | |
Respondents were asked if they intended to use the information provided by the HSL at some point in the future, regardless of whether there was an immediate impact or if their question was answered (Table 6). All respondents (54) indicated they intended to use the information in the future. Thirty (55.6%) respondents said the information would change their approach to future patients, whereas 15 (27.8%) said the information would change their approach to a particular patient. Thirty-seven (68.5%) respondents reported they would share the information with colleagues, 23 (42.6%) said they would use the information in their teaching, and 10 (18.5%) intend to use the information to revise clinical pathways, practice guidelines, policies, or procedures. Other responses given were: “to develop a nursing care plan”, “to keep knowledge current”, and “patient education to help with decision making”.
| Response | No. (%) |
|---|---|
| The information will not be used in the future | 0 (0) |
| Change approach to a particular patient | 15 (27.8) |
| Change approach to future patients | 30 (55.6) |
| Share/discuss with colleagues | 37 (68.5) |
| Teaching | 23 (42.6) |
| Research or publication | 6 (11.1) |
| Revision of clinical pathways, practice guidelines, policies, or procedures | 10 (18.5) |
| Other | 7 (13) |
| Total unique responses* | 54 |
| *Respondents could select all that applied; therefore, the column does not equal 100%. | |
The results of this study show that libraries can effect change in patient care. It is possible for hospital libraries to quickly assess the impact a service such as literature searching has on patient care. By rethinking how the RQHR HSL measures success it was possible for the library to transform its literature searching service measures into a clear picture of impact within the organization. Assessing impact can be accomplished without requiring much time or effort from practitioners asking questions or the library team responding to requests. The survey also confirmed the library's impression that the majority of patient care questions are related to the treatment or management of patients. By providing information on treatment or management of patients, the HSL helps determine, confirm, or even change the proposed drug or treatment the patient received. It is also important to note that, although not all questions have immediate impacts on patients, users clearly indicate the intent to use the information provided by the HSL to inform decisions and practice in the future for purposes whose impact was not measured in this study.
Building upon the experience of previous impact surveys and following key recommendations for survey design and implementation—specifically, sending survey invitations with the answer and offering a small incentive—allowed the HSL to achieve a good response rate while at the same time minimizing potential recall bias from respondents. Keeping the survey short and narrowly focused also made it easier for the investigators to analyze responses. Investing the time to validate questions and pilot the survey helped create a tool that fills a need for libraries who want to assess their own impact on patients.
An unintended result of the survey was discovering that the HSL is answering its users’ questions. This was not something the HSL had previously measured, and although the library was not able to answer all users’ questions, it is affirming to know that, with few exceptions, when an answer is not provided it is beyond the immediate control of the librarian conducting the search.
Because of the size of this study, the applicability of results is limited to within the HSL's pool of users asking patient care questions. In addition, the survey questions, while validated for RQHR, must still be assessed before use in other populations. Future directions for research include adapting the tool and methodology to evaluate impact in other purposes for literature search requests (e.g., teaching) or other library services (e.g., document delivery). The HSL may also explore using the data from the answers provided by respondents of this study to conduct a cost analysis for the library's literature searching service as a companion piece to the current study.
This research tool was developed and validated in such a way as to be useful for other libraries that wish to assess their own impact on patient care with a minimal amount of additional work. The use of the tool by other libraries would be the first step towards assessing library impact using a common set of questions and would ensure the ability to compare the performance of hospital libraries in organizations around the world. To that end, librarians are encouraged to use the survey questions in Appendix A and to contact the authors if considering use of the tool in their own organizations.
The authors thank: Julie DeGroot and Ali Bell who assisted with the design and administration of the survey; the RQHR Health Sciences Library and Archives staff who sent survey invitations, reminders and incentives to participants; the Canadian Health Libraries Association/Association des bibliothèques de la santé du Canada who provided financial support in the form of an unrestricted research grant.
Canadian Health Libraries Association/Association des bibliothèques de la santé du Canada Research Grant.
| 1. | Weightman A, Urquhart C, Spink S, Thomas R, National Library for Health Library Services Development Group. The value and impact of information provided through library services for patient care: developing guidance for best practice. Health Info Libr J. 2009;26(1):63–71. |
| 2. | Dalton M. Key performance indicators in Irish hospital libraries: developing outcome-based metrics to support advocacy and service delivery. EBLIP. 2012;7(4). |
| 3. | O'Connor P. Determining the impact of health library services on patient care: a review of the literature. Health Info Libr J. 2002;19(1):1–13. |
| 4. | Weightman AL.Williamson J. Library & Knowledge Development Network (LKDN) Quality and Statistics Group. The value and impact of information provided through library services for patient care: a systematic review. Health Info Libr J. 2005;22(1):4–25. |
| 5. | Johnson M. The library as a resource for decision making in mental health care. Psychiatr Serv. 1995;46(5):493–5. |
| 6. | Klein MS, Ross FV, Adams DL, Gilbert CM. Effect of online literature searching on length of stay and patient care costs. Acad Med. 1994;69(6):489–95. |
| 7. | Quality Development Team, Library Services Team at St. Michael's Hospital, St. George Hospital. A survey to assess the impact of mediated / facilitated literature searches on patient care. Stafford: South Staffordshire NHS Trust;2003. |
| 8. | Burton JE. The impact of medical libraries and literature on patient care in New Zealand. Bull Med Libr Assoc. 1995;83(4):425–30. |
| 9. | Brettle A, Hulme C, Ormandy P. Effectiveness of information skills training and mediated searching: qualitative results from the EMPIRIC project. Health Info Libr J. 2007;24(1):24–33. |
| 10. | Lindberg DA, Siegel ER, Rapp BA, Wallingford KT, Wilson SR. Use of MEDLINE by physicians for clinical problem solving. JAMA. 1993;269(24):3124–9. |
| 11. | Marshall JG. The impact of the hospital library on clinical decision making: the Rochester study. Bull Med Libr Assoc. 1992;80(2):169–78. |
| 12. | Marshall JG, Sollenberger J, Easterby-Gannett S, Morgan LK, Klem ML, Cavanaugh SK, et al.. The value of library and information services in patient care: results of a multisite study. J Med Libr Assoc. 2013;101(1):38–46. |
| 13. | Marshall JG. Measuring the value and impact of health library and information services: past reflections, future possibilities. Health Info Libr J. 2007;24:4–17. |
| 14. | Medernach C, Franko J. Assessing the impact of information services in a regionalized health-care organization. Health Info Libr Jour. 2007;24:46–56. |
| 15. | Naeem S, Bin, Ahmed S, Rabbani M, Waqar. The impact of hospital libraries on patient care and clinical decision-making: a survey of teaching hospitals. J Hosp Libriansh. 2013;13(2):120–30. |
| 16. | Urquhart C, Davies R. EVINCE: the value of information in developing nursing knowledge and competence. Health Libr Rev. 1997;14(2):61–72. |
| 17. | Urquhart C, Thomas R, Ovens J, Lucking W, Villa J. Planning changes to health library services on the basis of impact assessment. Health Info Libr J. 2010;27(4):277–85. |
| 18. | Cuddy TM. Value of hospital libraries: the Fuld Campus study. J Med Libr Assoc. 2005;93(4):446–9. |
| 19. | Bryant SL, Gray A. Demonstrating the positive impact of information support on patient care in primary care: a rapid literature review. Health Info Libr J. 2006;23(2):118–25. |
| 20. | Hurrell C, Aitken E, Lin Y, Vaska M. Evaluating the impact of a hospital library literature search survey [poster presentation]. J Can Health Libr Assoc. 2014;35(2):107–8. |
| 21. | Konrad SL, Boyce E, McKeown S, McTavish J. Survey design for evaluating librarian-mediated literature searches at a large teaching hospital [poster presentation]. J Can Health Libr Assoc. 2014;35(2):97. |
