Usability Study Identifies Vocabulary, Facets, and Education as Primary Primo Discovery System Interface Problems

Abstract

A Review of:
Brett, K. R., Lierman, A., & Turner, C. (2016). Lessons learned: A Primo usability study. Information Technology and Libraries, 35(1), 7-25. https://doi.org/10.6017/ital.v35i1.8965

Abstract

Objective – To discover whether users can effectively complete common research tasks in a modified Primo Discovery System interface.

Design – Usability testing.

Setting – University of Houston Libraries.

Subjects – Users of the University of Houston Libraries Ex Libris Primo Discovery System interface.

Methods – The researchers used a think aloud usability test methodology, with participants asked to verbalize their thought processes as they completed a set of tasks. Four tasks were developed and divided into two task sets (Test 1 and Test 2), with session facilitators alternating sets for each participant. Tasks were as follows: locating a known article, finding a peer reviewed article on a requested subject, locating a book, and finding a newspaper article on a topic. Tests were conducted in front of the library entrance using a laptop equipped with Morae (screen and audio recording software), and participants were recruited via an assigned “caller” at the table offering library merchandise and food as a research incentive. Users could opt out of having their session recorded, resulting in a total of fifteen sessions completed with fourteen recorded. Thirteen of the fifteen participants were undergraduate students, one was a graduate student, one was a post-baccalaureate student, and there were no faculty participants. Facilitators completed notes on a standard rubric, coding participant responses into successes or failures and noting participant feedback.

Main Results – All eight participants assigned Test 1 successfully completed Test 1, Task 1: locating a known article. Participants expressed a need for an author limiter in advanced search, and had difficulty using the citation formatted information to locate materials efficiently. Again, all eight participants found an article on the requested subject in Test 1, Task 2, but two were unable to determine if the article met peer review requirements. One participant used the peer-reviewed journals facet, while the rest attempted to determine this using the item record or with facilitator help. All seven participants in Test 2 were able to locate the book requested in Task 1 via title search, but most had difficulty determining what steps to take to check that book out. Five participants completed Test 2, Task 2 (finding a newspaper article on a topic) unassisted, one completed it with assistance, and one could not complete it at all. Five users did not notice the Newspaper Articles facet, and no participants noticed resource type icons without facilitator prompting.

Conclusions – The researchers, while noting that there were few experienced researchers and a narrow scope of disciplines in their sample, concluded that there were a number of clear barriers to successful research in the Primo interface. Participants rarely used post-search facets, although they used pre-search filtering when possible, and ignored links and tabs within search results in favour of clicking on the material’s title. This led to users missing helpful tools and features. They conclude that a number of the usability problems with Primo’s interface are standard discovery systems usability problems, and express concern that this has been inadequately addressed by vendors. They also note that a number of usability issues stemmed from misunderstandings of terminology, such as “peer-reviewed” or “citation”. They conclude that while they have been able to make several improvements to their Primo interface, such as adding an author limiter and changing “Peer-reviewed Journals” to “Peer-reviewed Articles”, further education of users will be the only way to solve many of these usability problems.