Evidence Based Library and Information Practice

Using Evidence in Practice

 

A Mixed Methods Approach to Iterative Service Design of an In-Person Reference Service Point

 

Kyla Everall

User Services Librarian

University of Toronto Libraries

Toronto, Ontario, Canada

Email: kyla.everall@utoronto.ca

 

Judith Logan

User Services Librarian

University of Toronto Libraries

Toronto, Ontario, Canada

Email: judith.logan@utoronto.ca

 

Received: 10 July 2017    Accepted: 6 Oct. 2017

 

 

cc-ca_logo_xl 2017 Everall and Logan. This is an Open Access article distributed under the terms of the Creative Commons‐Attribution‐Noncommercial‐Share Alike License 4.0 International (http://creativecommons.org/licenses/by-nc-sa/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one.

 

 


Setting

 

The University of Toronto is Canada’s largest university, with an enrolment of over 88,000 students (University of Toronto, n.d.). Currently ranked fourth by the Association of Research Libraries, with an annual total expenditure of $72 million USD (ARL, 2016), we have 44 libraries spread over three campuses.

 

Problem

 

Our largest library is a 14-floor building located in the center of our downtown campus.

 

When John P. Robarts Library was first opened in 1973, the reference desk on the fourth floor was conveniently located next to the card catalogues and near the main entry point to the book stacks. The card catalogues are long gone and the ground floor is now open for stacks access, but the reference desk has remained nearly unchanged physically. Likewise, our service model remained stable for decades. Two or three library staff members sat behind a large wooden desk from mid-morning to early evening. Shifts were primarily staffed by librarians with support from student librarians trained in the department. Our schedule called on librarians regardless of specialized expertise; when our department incorporated government information librarians in 2009, they, too, served on the reference desk.

 

 

Figure 1

“John P. Robarts Library” by Jeff Hitchcock on Flickr. Retrieved from https://www.flickr.com/photos/arbron/29821692490

 

 


 

Over the last ten years, student feedback collected through LibQUAL+ and other means has identified that Robarts is a confusing and intimidating library to navigate. Service provided by staff throughout the building did not always meet students’ expectations. We partially addressed this longstanding problem in 2015 when we added a roving service of student library assistants whose primary roles were to direct users to the correct location within the building and to troubleshoot problems with self-service machines such as printers. The usage statistics for this service showed the most traffic on the second floor of the building. This made sense considering that there were no library service points or staff members on this floor besides one entrance monitor who monitors the security gates and enforces our food policy.

 

We hoped that redesigning our outdated reference desk could also address gaps in Robarts’ perceived user-friendliness.

Evidence

 

We approached this problem within a service design framework. A relatively new method in libraries, service design is a “holistic, co-creative, and user-centered approach to understanding customer behavior for the creation or refining of services” (Marquez & Downey, 2015, para. 7). The aim is to use lightweight research methods to gain insights and make decisions, rather than conduct exhaustive studies of the problem (Polaine, Løvlie, & Reason, 2013). Service design also encourages prototyping, allowing service providers to test their ideas without a large investment of resources (Polaine et al., 2013).

 

Looking at our reference desk statistics emphasized just how much usage patterns have changed over time. Our interactions declined by more than 50% since the 2009-2010 academic year (Figure 2). Demand for the service was clearly changing. We asked ourselves, if in-depth reference expertise was only needed in about a third of the interactions currently on the desk, was there a better way to deploy our staff without sacrificing quality of service?

 

 

 


Figure 2

Reference desk interactions from 2009-2016.

 

 

Figure 3

Average estimated cost of reference desk interactions per month.

 

 

 


To investigate the effectiveness of our service model’s staffing mix, we did a rough cost analysis to estimate the cost per interaction (Figure 3). This analysis indicated that there were times of the year when our service model was more cost effective than others. We planned to use this information when designing the schedule of our redesigned service point.

 

 

Figure 4

An example of a completed journey map for a student who wanted to use scanners.

 

 

 


Further probing this issue, we conducted focus groups with our student librarians to assess their training and identify gaps in their knowledge. They reported high levels of confidence in answering general research questions. Referrals and questions that require organizational knowledge were their biggest challenges. This indicated to us that, once trained, our student librarians might be able to provide the bulk of staffing at our new service point if properly supported.

 

We also conducted a journey mapping research project to help us understand how our users were moving through the building. We surveyed users as they left the library about their goals during their visit. We then asked for a step by step breakdown of how they tried to achieve their goals and how they felt during each step (Figure 4).

 

We found only 35% of users came to the library with scholarly goals. This category included using print resources and working with librarians. Respondents reported two pain points that had previously surfaced in LibQUAL+: navigating the building and unhelpful interactions with staff.

 

Implementation

 

We moved our reference desk to the second floor of the building and rebranded it the “Ask us! desk” with a large sign placed next to it. All staff wore blue vests emblazoned with an information symbol while working on the service desk for the first eight months.

 

We purchased a variable height standing desk on castors with the intention of moving it around the floor to identify the ideal location, but we discovered that the wireless service was too unreliable. Instead, staff connected the desk’s laptops to a nearby LAN network. Happily, this was near the entrance to the cafeteria and a very busy coffee shop. We forwarded our reference desk’s extension to a cell phone so we could continue to offer telephone service, and issued earbuds to staff so they could be hands free while serving users on the phone.

 

A team collaboration and chat tool called Slack (https://slack.com/) was already in use in other units of our library, so we used it to facilitate communication between staff members at the Ask Us! desk and the rest of the department. We created a Slack “team,” which is similar to a message board, and invited as many public service staff members in the building as we could in hopes of increasing communication between service points.

 

Once we were physically set up on the second floor, we began to adjust our staffing mix. We stopped scheduling our government information specialist librarians on the desk as their expertise is in high demand on other reference channels. We also stopped scheduling two librarians at the same time; the maximum complement became one student librarian and one librarian. Once our student librarians were trained, on boarded, and sufficiently mentored, we moved to a backup model where the librarian assigned to the shift remained in their office and stayed in communication with the student librarian on shift over Slack. If the shift became busy or the librarian's expertise was needed, the student librarian would ask the librarian to come down to help over Slack.

 

Outcome

 

To see if the new model was working for our users, we conducted exit surveys in the fall and winter semesters. Respondents reported high levels of satisfaction with the service that they received at the desk (Figure 5), and 87% of respondents liked the location (Figure 6). Only 4% of respondents had a negative impression of the location, and the remaining 9% were either neutral or responded in a way that was not readily classifiable (e.g., “I wish we had a desk like this on every floor”). We also discovered that the blue vests were not significantly more important to our visibility than signage or staff demeanor (Figure 7).

 

 

Figure 5

Were you satisfied with the help that you received today? (n=77)

 

 

Figure 6

What do you think of the current Ask Us! desk location? (n=77)

 


Figure 7

What was the main thing that made you think that you could get help at this desk? (n=77)

 

 

Figure 8

Reference desk/Ask Us! desk interactions, 2009-2017.

 

 

 


We monitored our usage statistics to see what impact the new location would have. While we saw an overall increase in usage, the bulk of the questions were directional (Figure 8). Brief and in-depth reference questions declined more than in past years, suggesting that we need to reassess the new model to ensure that it is attracting in-depth reference appropriately.

 

We also repeated the journey mapping exercise to gauge if the new desk had any impact on users’ holistic experience of the library. There was almost no change in the majority of pain points reported, but this time there were no complaints about library staff. There are too many variables to attribute this positive outcome to solely our service, but it is an encouraging sign.

 

Throughout the implementation process, we consulted the staff working on the desk about the new model. Some staff voiced concerns about the blue vests. They felt strongly that wearing the vest made them look unprofessional, and reported that staff from other units had made comments to this effect. Since we learned from the exit surveys that the vests were not significantly more important to identifiability than other factors, we replaced the vests with lanyards.

 

Librarians reported noticeable improvements in their mentoring relationships with student librarians. Due to the layout of the new desk, we worked shoulder to shoulder with them, creating more feelings of teamwork and more collegiality than in previous years. We also noticed that the student librarians’ skills developed significantly faster under the new model.

 

Reflection

 

By considering a variety of data sources when redesigning our service point, we developed a model that better fit our users. Our intention was always to do just enough research to allow us to produce a prototype which we could refine after the initial implementation. When we encountered unforeseen obstacles, such as poor Wi-Fi, we reconfigured the service. Rather than derailing the project, we were prepared to address issues as they arose, ultimately resulting in a more responsive, flexible service.

 

Prior to this project, the prevailing organizational narrative was that service changes involving staff would be met with overwhelming resistance. A benefit of launching our new service model as a work in progress was more staff engagement and less resistance than anticipated. This prototyping approach signaled to staff that their feedback was not only tolerated, but necessary to the success of the project.

 

 

References

 

Association of Research Libraries. (2016, August 14). Spending by university research libraries, 2014-15. Retrieved from http://www.chronicle.com/interactives/almanac-2016?#id=65_416

 

Marquez, J., & Downey, A. (2015). Service design: An introduction to a holistic assessment methodology of library services. Weave: Journal of Library User Experience, 1(2). http://dx.doi.org/10.3998/weave.12535642.0001.201

 

Polaine, A., Løvlie, L., & Reason, B. (2013). Service design: From insight to implementation. Brooklyn, N.Y.: Rosenfeld Media.

 

University of Toronto. (n.d.). Quick facts. Retrieved from https://www.utoronto.ca/about-u-of-t/quick-facts

 

 






Evidence Based Library and Information Practice (EBLIP) | EBLIP on Twitter