‘ ‘ We want it now and we want it easy ’ ’ : Usability testing of an online health library for healthcare practitioners

Introduction – The purpose of this study was to undertake website usability testing of the Saskatchewan Health Information Resources Partnership (SHIRP) online library website,. a digital library for healthcare providers working in the province of Saskatchewan, to determine whether the SHIRP website is intuitive for healthcare practitioners to use. Methods: Thirteen volunteers from four locations in the province participated in a usability test that included a portion devoted to the completion of tasks, as well as a series of semi-structured interview questions. Data were analyzed and themes were identified that were used to redesign the SHIRP website. Results – Nine out of the 13 main menu terms on the SHIRP website were problematic. A relatively low number of participants completed the assigned tasks on the first try. The SHIRP website was determined to be unwieldy and not completely intuitive. Conclusions – Asking front line healthcare providers what they need and want in an online library website should be the first step in creating or redesigning such a site. The time available to healthcare providers for doing library research is often limited, so the site needs to be simple, clean, and fast to use.


Introduction
The Saskatchewan Health Information Resources Partnership (SHIRP) is an online special library based at the University of Saskatchewan that consists of a suite of authoritative electronic health information resources.More than 6000 full text electronic journals, 144 electronic books, and 15 databases are licensed by SHIRP for use by all employees of Saskatchewan's 13 health regions, all healthcare providers in the province, faculty and students at all postsecondary institutions in the province, and all Saskatchewan provincial government departments related to health.There are approximately 26 000 -30 000 potential SHIRP library users in the province.
Two of the 13 health regions, all of the postsecondary institutions, and at least one government department have their own libraries with their own points of internet access through which the SHIRP resources are made available.However, for the remaining health regions, all the independent healthcare practitioners (i.e., those not employed by a health region), and the remainder of the units in government departments related to health, the SHIRP website (www.shirp.ca) is the access point for the online library (see Fig. 1).
Rolled out in 2005, the SHIRP website had never undergone usability testing.The site was created by the SHIRP Outreach Services Librarian and the SHIRP Coordinator, with input from the two SHIRP co-chairs, and has shifted from html coding to two different content management systems (CMS): first Plone and then Drupal.The CMS for the current SHIRP site is Drupal, chosen for its robust functionality, adaptability, and the fact that it is an open source product.Over time, conversations often occurred about layout, terminology, functionality, and usability.Better late than never, we decided to head out into the province and see what the actual users of the site could tell us.We asked the following question: ''Is using the SHIRP website intuitive for healthcare practitioners in Saskatchewan?''Intuitive, in this case, is defined as the website users being able to use knowledge of how websites are designed in general terms (layout, terminology, etc.) to be able to navigate the SHIRP website with relative ease, and to find what they are looking for.We measured intuitiveness by the number of subpages a user visited before finding the correct page or giving up on the task.
A review of the literature reveals a lack of published usability studies that specifically focus on library websites used by healthcare providers in a practice setting; however, there are many published studies that deal with academic health sciences library websites and articles about usability studies in general that provide insight into the process.These articles generally discuss three aspects of usability studies: recommendations for website design, how to conduct a usability study, and user behaviour.

Recommendations for website design
It is not possible to create a website that will suit all tastes [1,2], but generally speaking, simple and "moderately complex" sites are preferred to complex websites [2].While simplicity is ideal, functionality remains key.For example, Norris found that simplifying library users' experiences by incorporating federated searching into the Michigan eLibrary website was in line with what their patrons asked for; one simple, Google-style box for searching everything, but patrons were not satisfied with the resulting functionality, as the site did not give them effective content retrieval [3].
Fox argues that library websites are often designed as a substitute for library staff, providing much of the same information without the reference interview [1].This argument is particularly relevant in situations such as the one encountered by SHIRP, where patrons cannot opt to get help in person when they need it.Library instruction sessions on how to navigate the website cannot be relied upon to make up for poor site design, because it is not possible to train all SHIRP users, and a large number of our patrons are occasional users who may forget what they have learned at a training session before they have the chance to cement that knowledge.The absence of personal interaction when patrons are using a site makes site design key to patrons immediately getting a sense of where they are and where they need to go on the site to make use of the library resources [1].
The terminology used on a library website is an important aspect of site design.Kupersmith has documented a variety of terms that do and do not contribute to library website usability [4].One strategy that appears to work well is making a library website task oriented, using terminology like "find an article" rather than using labels like "databases" or "e-journals" that are unfamiliar to many library patrons [5].While aspects like terminology or the font used on a site might seem minor, it is small things that will make the difference between users either giving up in frustration or making effective use of the site content [6].

How to conduct a usability study
Various methods can be used to evaluate a website: examining help-desk call logs, gathering impressions from help-desk staff as to what users have the most difficulty with, getting expert feedback on the site, conducting focus groups, and conducting usability testing [7,8].
Many articles cover the ''how to'' of conducting usability testing and provide a model for setting up a usability study.They often recommend asking approximately five participants, more if the group is diverse, to think aloud as they complete tasks that are designed to test the site rather than to test the participants' information literacy skills.These studies also frequently make use of facilitator-observer pairs, where one investigator guides participants through the session and prompts them to share their thoughts as they complete the c11-024.3d13/9/11 17:06:39 Fig. 1.A screen shot of the SHIRP website prior to the redesign.tasks, while another investigator observes the session [5,[7][8][9][10][11][12].However, there are some other factors to consider that go beyond this basic formula.
When designing a usability study with remote users, the first factor to consider is whether the tests will be conducted at a distance using special software, similar to webconferencing software that participants can download to their computer, or whether the tests will be conducted in person.While the first option offers the potential benefit of reduced expenses for travel and staff time, there are also potential drawbacks to this method.A certain level of computer savvy and comfort with this type of technology is required of participants, which may or may not be present in the user group.It is also common for healthcare facilities to have heightened network security featuring firewalls and restrictions on the types of applications that can be utilized [13], so participants who are located in a tightly controlled network environment may not have the ability to download the required software.Conducting usability testing at a distance also prevents the investigator from reading participant body language and witnessing the site being used in a ''real'' environment [8].
Another factor to consider when designing any usability study is the role of the investigators during usability testing.Sonderegger and Sauer have demonstrated that the investigators themselves can play a role in how well participants perform their assigned tasks.Participants may do better when the facilitator establishes a rapport that puts them at ease, while the presence of a ''non-interactive observer'' can unnerve participants, resulting in poorer performance [14].Alshammari and Mayhew have also noted that investigators can impact study results by bringing their own biases to the process in what they call the ''Evaluator Effect''.The Evaluator Effect comes into play during the formation of the tasks as well as during the interpretation of the results.While it cannot be eliminated, efforts can be made to reduce the Evaluator Effect, such as discussing the results with other evaluators or arranging for a third party to analyze the results [15].

User behaviour
One of the benefits of the usability testing process for library websites is that it allows investigators to become better acquainted with library patrons and how they behave when using the library resources.Even librarians who believe they know their users may be surprised at what they find.Emde et al.'s [4] study of a redesigned library website with faculty and graduate students found that 90% of study participants were resistant to change, and found ways around the modifications to the website to continue using it in the same way that they had before the redesign.They also found that 80% of participants said they would contact a librarian if they needed help, rather than look for help on the website.Perhaps their most interesting finding is that when asked to discover if a particular journal article was available, half of the participants searched for the article title or the author, rather than the journal title.At times they would use inappropriate search tools, such as the online catalogue or the search of database titles, which implies that articles are seen as stand-alone items, rather than pieces of a whole (i.e., the journal) [4].
While some people are naturally curious and willing to explore a website to find what they need [1], others are not prepared to invest time and effort in navigating a website.Participants in Dougan and Fulton's test of a redesigned library website at the University of Illinois Urbana-Champaign, Champaign, Illinois, would scan the site for particular terms on the page; known as ''trigger words'' [16], and did not consider their options before choosing where to go to complete their task.Instead, they would stop at the first instance of a word, even if it was not what was needed to complete the task at hand.They also discovered that some patrons had missed information included on the previous version of the website because they had never scrolled down the page [9].Rosenbaum's [10] usability study of The Cochrane Library reinforces the idea that library users, healthcare providers in particular, are not interested in expending any more effort than necessary: ''Repeatedly we heard praise for the quality of content of this site.But frustration levels were very high, and several participants said they were ultimately too lazy to bother to use a site that made it so difficult for them" [10].Conversely, Norris found that for the Michigan eLibrary website "more advanced users were happier users", likely because these individuals regularly use the library website, and had become accustomed to where things were located and what must be done to get results [3].These findings suggest that something as simple as the website's primary audience; advanced users who frequent the site, or less sophisticated, occasional users, is an important factor in site design that may inadvertently be overlooked.

Methods
The authors created a two-part usability test that incorporated user tasks and open-ended follow-up interview questions to guide the discussion.The study was created after looking at the usability literature, consulting with a health region library that had recently completed a usability study, and examining the current SHIRP website for what we suspected were problem areas based on what we knew of the patrons' use of the site.We designed the test so that participants would complete tasks that we believed would be typical of a SHIRP library patron and that had posed a challenge to some patrons in the past.This study was declared exempt from formal ethics review by the University of Saskatchewan Behavioural Research Ethics Board; however, ethical principles and procedures for research were adhered to.The usability test was pre-tested with three volunteers from the University of Saskatchewan Library chosen out of convenience: two librarians and a library staff member.
The list of tasks participants were asked to undertake are as follows (see Appendix A for the complete SHIRP usability study including tasks and questions): The pre-test resulted in no modifications to the tasks or questions, but it did confirm that one author was better suited to be the test facilitator and the other author was better suited to be the observer.
We followed the lead of Jakob Nielsen, well-known usability guru, when deciding on the number of people to involve in our usability testing.Nielsen's research revealed that if the user group is fairly homogenous, then no more than five participants are needed [12].If the user group includes several distinct groups of users, then more participants are included.Because SHIRP users encompass healthcare providers at a variety of levels, as well as some users who are in health fields but who are not healthcare professionals, we decided to aim for 15 users for our usability test.Volunteers were solicited by contacting people familiar to the SHIRP organization to see if they, or someone in their organization, would be willing to participate.E-mail canvassing resulted in a number of volunteers coming forward.Fourteen volunteers were recruited.One participant failed to keep the testing appointment.Ultimately, 13 study participants, primarily from smaller centres in the province (10 healthcare providers, two employees of a health quality organization, and one library technician working for a health region) were asked to complete a pre-determined set of seven tasks using the SHIRP website while thinking aloud.The think-aloud method involves having participants speak about their thought processes while they are completing the tasks.They are asked to not only report what they are doing, but also, to describe why they are choosing a certain link or pathway for task completion.Afterwards, participants were asked to share their thoughts and opinions about the website in a semistructured interview (see Appendix A).
Sessions were held at the volunteer's place of employment with the participant, the tester, and an observer who took notes.The observer interacted with participants at times, responding when the participants asked a question or had a comment in both portions of the test, and asking for more information relating to their performance of the task.The participation by the observer served to elicit more thorough feedback and, according to the literature, put the participants more at ease than if they were faced with a silent observer.An audio recording was made of each session using a small digital voice recorder.At the end of each session, as a token of appreciation for their participation, each participant was presented with a $10 Tim Horton's gift card.
Each usability test followed the same pattern.We greeted the participant and outlined what the session would entail.Formal introductory material was presented using a prepared script, with care taken to explain that this session was not to test participants but to test the website.The participant signed the consent form, which was forwarded for perusal prior to the session.The participants completed the tasks on the SHIRP website, with 7 minutes allowed for each task.Tasks were finished when the participants felt that they had completed them, or, if more than 7 minutes had elapsed, the task was stopped.The observer (note taker) was also the time keeper.Quite often participants used the opportunity of the usability test to learn more about the SHIRP resources and wanted to know the ''right'' answer.If appropriate, we showed participants how to use certain resources or where to find certain items on the website once a particular task was finished.One participant in particular signed up for the usability test specifically to learn more about SHIRP.
Data were collected on where the participants navigated on the website and what they said in the think aloud portion.Transcripts were not made of entire sessions.Instead, the audio recordings were used as confirmation for data analysis.The observer transcribed her notes for each session.
Our data analysis was informed by grounded theory, where ''theories are allowed to emerge from data, as opposed to previously formulated hypotheses which are 'tested' against data'' [17].We examined the data separately and then compared results in an attempt to minimize bias as recommended by Alshammari and Mayhew.When we disagreed on aspects of a participant's performance, for example, on how many steps it took for a participant to complete a task, we reviewed the data and came to a consensus.We began with whether or not the participant was successful in completing the task, and if so, if it was on the first try.As various participants explored different parts of the website, their movements were tracked by adding headings to a spreadsheet.Headings for the locations visited were constructed as the analysis progressed.For example, the second task asked participants where they would go to find a video on chest tube insertion.The first participant started by going to the link for multimedia, thus the heading ''multimedia'' was placed on the spreadsheet.The next participant started by going to video tutorials, which was also added as a field.The semi-structured interviews held after the task portion of the testing were analyzed to tease out themes by which to focus web redesign efforts.The notes were combed for recurring issues and problem areas, supported by revisiting the audio recordings of the sessions.

Results
Our participants had varying levels of success with the tasks they were asked to perform.With the exception of Task Four (looking for the specific journal article) and Task Six (where to find Medline), relatively few partici-c11-024.3d13/9/11 17:06:40 pants were able to navigate to the correct section of the website on their first attempt.Some participants only visited one place on the website; however, they were not always successful.For each task there was at least one participant who visited four or more areas of the website before finding the correct location or giving up on the task (see Table 1).We decided not to measure the participants' success with Task Three -Where would you go for help on how to use the resources available on the SHIRP website?during data analysis because there was more than one ''correct'' answer, including offline options such as consulting with an individual who would know what to do.We did not eliminate the data related to this task even though there was more than one right answer because the results provided useful information.
Analysis of the notes taken during the session, supported by the audio files of the session recordings, identified five important themes surrounding participants' ability to successfully navigate the website.

Theme 1: Where's the search box?
There was an overwhelming sentiment expressed by all participants that the website needs a search box as a means to quickly search the resources (i.e., federated search) and to search the website itself to locate a particular resource.A preference for Google's search functionality was mentioned many times by participants.

Theme 2: Terminology
The terminology used on the website, including 9 of the 13 main menu items, confused the majority of testing participants.These problematic terms included the following: Participants suggested using more self-explanatory terms and alt tags (html code used to display a short text description of a link when the mouse is hovered over it) that provide more explanation of a menu item to help with ease of use and more intuitive navigation.We also found that while there were numerous options on the site for finding help; including contact information, handouts used in training sessions, and video tutorials, the absence of the term ''help'' caused much confusion among the participants.

Theme 3: Website organization and design
Participants indicated that the website's organization was suboptimal.The main menu was deemed to be unwieldy, and several participants indicated that this problem might be mitigated by grouping similar functions under larger headings.For example, participants suggested placing links to pages featuring video tutorials, handout material, and contact information under an overarching ''Help'' section, and featuring links to the e-journals list, the databases page, and the interlibrary loan document request form (''request an article'') on a single ''articles'' page.
In terms of visual effectiveness, the participants called for better use of colour, headings, and font to help them distinguish between the resources listed on the page as well as the pages themselves, as the individual pages on the site seemed to blur into one another.Participants also indicated that the pages should be revamped to make better use of space, reduce the amount of text present, provide better resource descriptions and, where possible, minimize the amount of scrolling required.Specific recommendations included freeing up valuable real estate that was occupied by the SHIRP banner, and adjusting the login area, making both of these elements smaller and less obtrusive.

Theme 4: Time
Approximately half of the participants mentioned time when using the website.The contexts varied, from too much clicking to get where they needed to go, to the webpages loading too slowly, to the amount of time healthcare providers have to look around the website to find what is needed.

Theme 5: Most important part of the SHIRP website
When asked what the most important part of the website was, the participants indicated that the databases were most important (mentioned five times), followed by e-journals and articles (mentioned four times each); the left hand menu (mentioned three times); multimedia (mentioned two times); and the About SHIRP page (mentioned once).c11-024.3d13/9/11 17:06:40

Discussion
Many insights from the library literature regarding user behaviour and websites, such as use of trigger words, an aversion to scrolling, and viewing journal articles as separate entities, also emerged in this study.The terminology used on the SHIRP website was clearly a problem for SHIRP users with 9 out of the 13 main menu items included on the list of problem terms.We had anticipated the standard library jargon issues, such as featuring a link labeled as ''databases'' instead of ''search for articles'', but we were surprised to learn that things we thought were obvious were not.Examples include ''request an article'', which was the link to our ILL form that was misinterpreted as the place to go to request a literature search.''Multimedia'' was another problematic term; many participants gravitated to the video tutorials, which describe database searching, when asked to find a video on chest tube insertion, simply because they saw the word ''video'' the title.One participant also told us that she associated the term multimedia with advertising media.
As previously mentioned, session participants had a great deal of trouble with the task ''Show us where you would go if you needed help using the website'', largely because they were looking for the word ''help'' and could not find it, but also because there were terminology issues surrounding ''Video Tutorials'' and ''Handouts''.Many websites feature a ''help'' link; perhaps if we had asked where they would look for assistance, participants would not have fixated on finding a link with a ''help'' label.The most obvious source of help was ''Contact Us'', and several participants noted the link to ''Contact Us''.However, unlike Emde et al.'s [4] patrons, who would contact their liaison librarian for assistance, our participants indicated that they would not want to call or e-mail unless it was their last resort (and sometimes not even then).The SHIRP office does receive help requests periodically, but the number of calls is less than might be expected given the large number of patrons we serve, which suggests that this hesitation may be widespread.Some study participants indicated that they would ask colleagues in their health region for help, while others indicated that the website itself should give them a better idea of where they needed to go and what they needed to do, eliminating the need to seek help at all.
Changes to the way information is organized and presented were needed to enable users to take in what is being presented at a glance, and make sense of what they find.We believe that changes such as condensing the main menu by grouping similar sections together, as in the case of sources of help, will make the site more intuitive and reduce the amount of time needed to puzzle out where to go and what to do.The popularity of a quick and easy search box cannot be denied; however, we are not prepared to make one available at this time.We currently face technical considerations that prohibit the creation of a functional site search, and we do not believe that a satisfactory federated search is currently available.
The success rate of our study participants is troubling.The number of participants able to successfully navigate to the correct area of the website to complete a given task on their first attempt was low, and out of those who were successful in the end, many guessed, used process of elimination, or clicked on random links until they found the page they needed.Some participants indicated that they were second guessing themselves as the session progressed; what they thought was the correct place to go for the previous task was wrong, so they did not trust themselves for fear of being wrong again.In our testing environment the participants knew there must be a way to complete the tasks we asked of them.Would they have continued on in their search for the right page in a normal situation, or would they have given up and moved on to Google, or simply gone without?In more than one instance, there were participants who implied that they would not persevere.
Usability testing revealed a great deal about how SHIRP patrons use the library, and how they think of library resources.We found that, as in the literature, most participants did not think of journal articles as pieces of a whole, and the idea of working through various layers of databases and journal titles to get to an article was confusing.Some participants recognized that when approaching something new, exploration and experimentation are necessary to deal with the learning curve involved, and we did find that participants used some of what they had learned while going through the website to complete tasks later on.But not surprisingly, more often than not, participants told us that healthcare providers ''[…] want it now and want it easy''.
Usability issues aside, many participants indicated that the library resources available through the SHIRP website were valuable, and access to them was much appreciated.But at what point does the frustration involved in accessing online information resources outweigh the benefit?In Rosenbaum et al.'s [10] examination of The Cochrane Library's interface, knowing that it was "the best" was not enough to make healthcare practitioners go through the pain involved in actually using it.If something like The Cochrane Library -the gold standard in evidence based medicine -could not get away with being difficult to use, what hope would our website, or any health library's website, have?The resources themselves can be difficult to use, so simply getting to them should not be an added obstacle.

Website improvements
We redesigned the SHIRP website based on the insights gained from usability testing and launched the new site in June 2010 (See Fig. 2).We made the terminology more straightforward and task oriented by featuring the categories Find an Article, Look for a Book, and Look for a Video in the main menu.We also added a Help page that incorporated subpages featuring help documents, contact information, and frequently asked questions.
The login box was moved below the main menu and the SHIRP banner was redesigned to make better use of space.We condensed news items and included more direct access to the various means of accessing journal articles to make site navigation faster and easier.We also created a site map to help patrons find their way in lieu of providing a search box.

Study limitations
It is important to recognize any limitations in a research study.We pre-tested our tasks with library staff, rather than c11-024.3d13/9/11 17:06:40 healthcare providers, out of convenience and therefore the questions themselves may have been interpreted differently by our study participants.The second limitation of this study is that our participants were volunteers, and as a result, we did not have representation from all healthcare professions.Healthcare providers from the very small towns were not represented.In terms of technology, our participants all had highspeed internet access, and we know that slower dial up and satellite internet access is in use in the province.We provided instruction mid-test at the request of some participants.This could have affected performance on subsequent however, we thought giving participants an explanation when requested was the right thing to do.

Conclusion
SHIRP is an online library, and our users are in practice settings mostly located in rural, small town, and smaller city centres.Observing website use in the field is essential in order to create a site that is intuitive and useful for healthcare providers.The time available to healthcare providers for doing library research is often limited, so the site needs to be simple, clean, and fast to use.We found that conducting the usability tests where the participants work was very beneficial, as we not only could observe how the participants use the website, but we also got a firsthand look at how the website displays on different browsers, monitors of different sizes, and through different networks.Creating a website in a vacuum is ill-advised, and usability testing will now be a regular part of SHIRP maintenance.Every website is unique, but the basic framework of this study could be generalizable to other similar digital libraries.For personal use only.
. Please show us where you would go to find this article: Please show us where you would go if you wanted to use Medline.7. A co-worker has told you about an article on the Heart Health Survey that you'd like to read.Please show us where you'd go to find the article.
1. Please show us where you would go to look up journal articles on the topic of dietary supplements for patients with diabetes.2.Please show us where you would go to find a video on chest tube insertion.3.Where would you go for help on how to use the resources available on the SHIRP website?c11-024.3d13/9/1117:06:40 Neilson and Wilson 53 JCHLA Downloaded from pubs.chla-absc.ca by University of Saskatchewan on 10/06/11 For personal use only.4

Table 1 .
Summary of participant success in completing the seven tasks.