Using a customized search engine to address low health literacy: a program description

: In the provision of health information, it is the structure as much as the content of an information source that can influence a patient’s ability to understand and act upon medical guidance. Canada has a commitment to providing equal, quality health care to all citizens, but a barrier to this equality has emerged as health information has gone online. This paper discusses the online information retrieval system in a patient library in a Canadian hospital, the barrier that this system created in connecting patients with health information, and how resources were reorganized to more equitably meet the needs of patients. The example of one information retrieval system provides a new frame through which to evaluate the organization of electronic information in health care settings.


Introduction
This paper outlines the construction of a customized health search engine to meet the health literacy needs of hospital patients and their families in the Patient and Family Learning Centre (PFLC) at St. Michael's Hospital in Toronto, Ontario. The system described is a custom search engine that is organized to allow users with low health literacy to search web-based resources for trustworthy, health-related information. As a volunteer at the reference desk of the PFLC during the summer of 2011, I was provided with training in health literacy and adult education, and asked to help PFLC users navigate the centre's collection of print and digital resources. As I spent time conducting searches with users, it became apparent that the information organization on the PFLC website did not meet the information needs of the population served by St. Michael's Hospital. Using Google Custom Search, I created an example of how the same resources could be reorganized to better serve user needs, and presented this example to the Patient Education Specialist responsible for the PFLC. Together with the hospital's health sciences library, the PFLC team worked to adopt this reorganization and integrated a customized search feature into the PFLC's website redesign. This paper discusses the invisible barriers to health literacy that were present in the original information retrieval model, and how these barriers were overcome through a new design.
Health information seeking is influenced by health status, cognitive skills, setting, and lifestyle [1]. These influences are particularly relevant when considering the health information behavior of groups with complex health needs, limited cognitive skills, and poor access to information sources. These characteristics are often a factor in the information needs of the inner city population served by St. Michael's Hospital. To address the influences that shape health information behavior, the information retrieval system at the PFLC was reconstructed using a definition of health literacy as a design and evaluative rubric. By using the definition of health literacy as a basis for evaluation, the PFLC team was able to better identify the knowledge assumptions present in the original information retrieval design, and was able to reorganize information retrieval in a way that better met user health information needs. Keyword searching was selected as the preferred method of information retrieval based on the observation of users. The decision to adopt keyword searching as an information retrieval solution on the PFLC ''Learn about Health'' website was supported by findings that health literacy is a significant barrier to patients' ability to access electronic health information and that greater patient understanding of electronic health information can be achieved by providing health specific search engines with query assistance and simple results presentation [1].

Literature review
The World Health Organization defines health literacy as ''cognitive and social skills which determine the motivation and ability of individuals to gain access to, understand, and use information in ways which promote and maintain good health,'' [2] but Zarcadoolas, Pleasant and Greer [3] have elaborated on this definition and identified four domains that contribute to health literacy: functional literacy and numeracy; science and technology E. Jurczyk. This author is a recent graduate of the Faculty of Information, University of Toronto.
(e-mail: eva.jurczyk@utoronto.ca). This paper is peer reviewed and the Login Canada Student Paper Prize winner. literacy; community and civic literacy; and cultural literacy [3]. The key skills identified within these four domains are comprehension of printed and spoken language, knowledge of fundamental health and science concepts, recognition of collective beliefs and customs, and knowledge of sources of information and how to interpret them within a civic system [3]. Zarcadoolas, Pleasant and Greer's facets of health literacy are a significant resource in understanding how patients perceive health information, and were therefore applied as an evaluative measure in the reorganization of patient information resources on the PFLC website. Since health literacy emerged as an area of study it has proven an accurate predictor of health outcomes [4] and as such, it has been suggested that health literacy needs should be considered in the production of health literature to ensure patients' ability to comprehend crucial health information [5]. Tailoring the presentation of health information can help close the gap between user health literacy and consumer health resources [1].
In a study of diabetes patients, Schillinger et al. [6] concluded that inadequate health literacy contributes to a disproportionate amount of diabetes-related problems.
Darren et al. [4] conducted a review of over 3000 articles and abstracts regarding health literacy and health outcomes, and found that patients who had been identified as having low health literacy were one-and-a-half to three times as likely to experience a poor health outcome. Schillinger et al. [6] and Darren et al. [4] both suggest it is necessary for the health community to intervene and provide resources and services that consider the results of health literacy studies.
Most patient-directed health information is written at a grade 11 to 14 reading level on the Flesch-Kincaid scale [7], while 48 per cent of Canadians do not have the reading comprehension skills to navigate complex blocks of text [8]. The complexity of the design of health information interfaces can also be problematic for users, even when the reading level of the text is appropriate. Complex tasks requiring multiple keystrokes are best suited for expert users; novice users are generally only able to complete tasks requiring few keystrokes or steps [9].
Little research exists surrounding the improvement of information retrieval in consumer health information applications [1] although term suggestion tools created for consumer health information seeking have shown some success in connecting consumers with relevant health information [10]. Though these term suggestion tools, or consumer health vocabularies, help overcome limitations of scientific literacy, they do not overcome psychosocial and cultural factors that limit user access to health information [10].
Cole, Cantero and Sauve [11] applied a model of information behaviour to define the needs of their users when constructing an information retrieval system for use by undergraduate students. To diagnose information needs, Cole, Cantero and Sauve used Kuhlthau's six-stage model of the information search process [12] to determine how users would be interacting with the information system at every stage of information retrieval. Kuhlthau's six stages, initiation, selection, exploration, formulation, collection and presentation [12], encompass the entire information retrieval process but it is the exploration phase that is of particular interest when considering how a search interface should be designed. At the exploration stage users must investigate information on the general topic in order to extend their understanding and narrow their search [12]. It is at this same stage that Kuhlthau has found users feel apprehension, even anxiety, potentially causing them to abandon their search [13]. As a response to research about information behaviour, Marill, Miller and Killendaugh suggest that a consumer health information interface should provide the opportunity for both searching and browsing to better facilitate topic exploration and alleviate user anxiety [14]. The MedlinePlus public user interface first employed an AÁZ index of topics as a form of rudimentary search engine, as this organization of information did not require user familiarity with Medline's knowledge organization [14]. MedlinePlus eventually made the change to keyword searching for information retrieval as their research showed users had come to expect a keyword search feature to allow them to search all website content [14]. MedlinePlus applied research conducted by the U.S. Department of Health and Human Services, which has laid out research-based guidelines for web design and usability, and concluded that users are best served by a feature that allows the entirety of a site's content to be searched [15]. These same research-based guidelines recommend that search functions should be adaptable to user vocabularies, integrate common misspellings and allow simple searches that do not require complex Boolean strategies [15]. Failure to include these features can result in user confusion and frustration; users will generally attempt only one or two searches before abandoning a search tool [15]. The usability research also found that interfaces should be designed to display critical information as close to the homepage as possible [16], as forcing users to click deep into the site can lead to incorrect choices and user confusion. This is particularly important in systems designed for novice users; the number of keystrokes or clicks needed to complete a task is a reliable measure of needed expertise for information system navigation [17].

Setting
St. Michael's Hospital is an inner-city hospital in Toronto, Ontario. Among the hospital's areas of expertise are critical care and trauma, heart disease, neurosurgery, diabetes, cancer care, and care of Toronto's vulnerable population [18]. Opened in the summer of 2011, the PFLC at St. Michael's hospital provides access to print and digital health resources for patients and their families. The PFLC is staffed entirely by volunteers, though information specialists in the hospital's Health Sciences Library are occasionally contacted to assist with more complex health queries. The PFLC provides computers for patient use, but restricts Internet access by removing the address bar on web browsers. Patients can only navigate the Internet through sites that are indexed on the PFLC ''Learn about Health'' website. Information organization presented a unique challenge for PFLC staff because the centre did not produce original content for the website, but rather provided links to consumer health resources that were deemed trustworthy and appropriate for a low level of health literacy. These sites were indexed, and links were provided from the ''Learn About Health'' website ( Figure 1).
Users were required to navigate the index of health information resources, select a resource, and then search within that website for the answer to their health query. Resources were sorted under the following headings: General Health Resources, Health and Wellness, Diseases and Conditions, and Diagnostics and Treatments. Within each of these areas, subheadings were listed alphabetically and within each subheading, a list of links related to that topic was provided ( Figure 2).
In the first weeks that the PFLC was open to the public, when volunteers were able to observe users interacting with the information arrangement, the index immediately revealed itself as problematic. Volunteers observed that users had difficulty situating their queries within the indexed list of sites, that users often did not have the correct spelling of the term they were looking for, and that users would abandon their searches, asking the volunteers to search on their behalf, or settling for information available in print resources at the PFLC.

Solution
To overcome the limitations presented by the indexbased information retrieval, the PFLC staff considered both the information behaviour and the health literacy of the user population. Focusing on Kuhlthau's exploration phase in her model of the information search process, which she identifies as the most difficult phase of information retrieval for users [12], the information retrieval interface was evaluated by applying health literacy literature to the action of attempting to formalize a health query and retrieve exploratory information using the PFLC website.
Google Custom Search was chosen as the tool to enable search on the PFLC website for several reasons. Limitations of time and budget meant that a free tool, which did not require complex code to be written, was a necessary choice. Google Custom Search is able to correct spellings and direct users to related search terms, as suggested by the usability guidelines of the U.S. Department of Health and Human Services, and offers advertising-free service at no cost to registered charities and non-profit institutions. Most importantly, Google Custom Search allowed PFLC users to search the content of every website indexed on the PFLC ''Learn about Health'' webpage from one search box embedded on the ''Learn about Health'' site ( Figure 3). The Google tool provides users access to the content of consumer health resources approved by St. Michael's Hospital, while limiting the information navigation skills, and keystrokes, required to get there.
When evaluated against the four tenets of health literacy, this method of information retrieval has many benefits. Functional literacy is the first dimension of health literacy defined by Zarcadoolos, Pleasant and Greer [3]; it is the ability to understand and use written and spoken text, and basic mathematical symbols [19]. Recognizing that functional literacy may be a challenge for many users, PFLC staff wanted to integrate audiovisual material into the information organization so that users could choose a non-textual way to access health information. Within the indexed system users had to choose a consumer health information resource, and then navigate within that resource to find videos. If the text within that resource was complex, or the videos not featured prominently, a user with low functional literacy would have difficulty accessing audiovisual content. Within the custom search solution PFLC staff added a ''Video'' filter, allowing users to enter a search term, select ''Video'' and retrieve a list of videos related to their query. The custom search tool further supports users with limited functional literacy by providing auto-complete and auto-correct, eliminating the need for precision in the spelling of search terms.
Scientific literacy is the second tenet of health literacy and is the ability to understand basic science and health concepts as well as common technology [19]. Consumer health vocabularies, like the one proposed by Zeng and Tse [10] are designed to overcome problems with scientific literacy by identifying and modifying references to biological systems and groups of diseases from complex scientific to popularly used language [20]. The PFLC did not have the resources to construct or apply a consumer health vocabulary as a search solution. Instead, low scientific literacy was overcome by applying a keyword search solution which would not require users to navigate the information organization to determine where their query would be situated, and used the popular Google search interface that suggests alternate or related terms, corrects spelling, and utilizes auto-complete to suggest search terms.
Cultural literacy, the third tenet of health literacy, is defined as the ability to recognize and understand cultural beliefs, norms and worldviews, as well as the ability to understand how those norms, world views and social identities shape and produce health information [3]. A strategy that has been used to overcome low cultural literacy is the dissemination of health information that contains material related to seemingly ubiquitous everyday behaviors such as eating, exercise, and sleep patterns [20]. The understanding of health information is shaped by cultural constructs including health belief models that may be magical or deterministic, priority identifications which may be familial and can cause individuals to value the family over self-care, and experiences of discrimination or stigma in health situations or environments [21]. These constructs may affect the way users perceive information organization in a health context, and may make navigation of an information index designed from a North American perspective difficult for some users. The custom search solution meant that users did not have to navigate an index and subject headings that may be organized quite differently from how users perceive health information. The search solution also allowed PFLC staff to integrate information resources that a user with low cultural literacy may not consider to be part of healthcare. For example, a search for the term breast cancer will retrieve not only  information about prevention and treatment, but also links to social support resources.
Civic, or community, literacy is the final tenet of health literacy, and is defined as knowledge about sources of information and services, and how to interpret those sources; civic literacy often includes media literacy and knowledge of government and civic processes [3]. Civic literacy governs an individual's ability to fill out necessary medical forms and to understand the relationships and roles of clinicians, administrators, and national, provincial, and municipal agencies [20]. A common complaint from users of the original health information index was the lack of clear demarcation of American material. There are many organizations, like the Mayo Clinic, producing high-quality consumer health information; excluding this information would be doing a disservice to users. The custom search solution allowed the addition of a filter that would show only Canadian content at user request, allowing users to better understand the role of clinicians, administrators, national, provincial, and municipal agencies within a Canadian healthcare context.

Outcomes
The PFLC and Health Sciences Library teams created the Google Custom Search by inputting the 100 URLs that were already indexed on the ''Learn About Health'' webpage into the Google tool, adding filters for videos and Canadian content, and creating a search interface that matched the St. Michael's hospital website style. The search box was first embedded only on the staff intranet, and used by PFLC volunteers when conducting searches on behalf of PFLC users. This two-month test period allowed staff and volunteers to identify any errors or omissions, and rectify them before the site became publicly available to users. After this initial test period, the search function became a prominent part of the PFLC ''Learn about Health'' website ( Figure 4).
The original index of health sites is still available, allowing users the opportunity to either search or browse for health information. By considering limitations of health literacy, the PFLC website can now serve as a valuable information resource for a broader range of users. The PFLC staff collects data about use of the centre and resources, including frequency of independent information retrieval versus instances when a volunteer assists users. With the passage of time, it will become quantifiable how the addition of a keyword search has impacted user ability to retrieve health information. In the short term, user reliance on volunteers for research assistance does seem to have decreased, and volunteer confidence when guiding users through the search process seems to have increased. At minimum, the addition of keyword search makes the PFLC website accessible to a broader range of users with variable levels of health literacy.
While observing patients interacting with the new custom search, several improvements in information retrieval were immediately observed. Patients looking for general information about a disease or condition, for example a form of cancer, were able to retrieve all applicable information in one search, and browse through all information resources related to that form of cancer in order to narrow their search. Previously this same query would have involved searching within the Diseases and Conditions, General Health Resources, and Diagnostics and Treatments headings. For a user at the moment of exploration who is attempting to formalize their query by investigating the general topic, the keyword search provided by Google Custom Search is a far superior approach at this moment of apprehension and stress. It has also been observed that the custom search tool is a superior solution for patients with a very specific query or information need in mind. For example, users frequently utilize the PFLC computers to access an online accessible transit order form. Within the index, the link to this order form fell under the heading of General Health Resources, and the subheading of Community Resources. Users generally struggled to find the link to this form in the index and in almost every case asked a volunteer for assistance. Using the custom search tool, once a user types the first three letters of the name of the service, the tool auto-fills the search box and retrieves the link to the form as the top search result. Though these examples of user interaction are not strictly related to user health literacy, it was by considering health literacy in the first place that ease of use was achieved.

Discussion
Framing the review of an information system through the scope of health literacy is a useful way to discern how well the system is fulfilling user needs, and if the system is excluding the needs of any users. A similar customized health search tool to the one described is provided by the Health on Net Foundation [22]. In the same way as the PFLC customized search, this portal utilizes only trustworthy information that is appropriate for a general patient reading level. Though the search portals are similar in this way, the advantage of an in-house solution is a focus on Canadian content, and the ability to curate content based on the service areas of St. Michael's hospital.
Despite our best efforts, there are limitations inherent in the PFLC custom search solution. Limited resources meant a pragmatic approach, and there are many aspects of a health information retrieval system that the team would like to see in the future that were not possible as part of this project. A consumer health vocabulary, one rigorously tested with users of many health literacy levels, will be a valuable component of any consumer health information retrieval system. As this was not available, keyword searching in combination with Google search algorithms was judged a sufficient alternative. More problematic is the fact that the custom search solution is not adaptable to various levels of health literacy. Instead, the system assumes a low level of health literacy, and the team hopes the information retrieved will meet the needs of most users. It is recognized that the information retrieved by the custom search tool will not meet the needs of some advanced users who are seeking information at greater depth, or with greater complexity.
Despite these limitations, the custom search tool on the ''Learn About Health'' website can be considered a success. By evaluating information retrieval using the things we know about health literacy, the team at the PFLC was able to determine assumptions that were being made about cultural knowledge, knowledge of the Ontario healthcare system, and basic scientific knowledge of users. Once these assumptions were made clear, we were able to determine how they were preventing users from successfully retrieving health information, and to make adjustments that provide users with better access to information.The new search tool built at St. Michael's hospital is an important step in providing resources for patients with limited health literacy, but it is not the last step. The search tool will be continually revised and updated to better serve user needs. The review of online patient resources through the frame of health literacy gives the team at St. Michael's an important way to evaluate resources in the future. The evaluation framework used to identify the limitations of the original information organization can be applied to future information structures that are intended for patient use, for example electronic patient health records.
Though the belief of the team is that a more user-centric system has been created, it is too early to comment with certainty on how the new system has affected the way patients interact with information on the ''Learn About Health'' website. The volunteers in the PFLC will continue to monitor use and make suggestions for further improvements. Overall, the team feels that the reorganization of health information from an indexed system, to one that allows keyword searching, is more inline with the principles of health literacy and provides better access to health information for all patients.