Low Volume, Funding, Staffing and Technical Problems are Key Reasons for Discontinuation of Chat Reference Services

Abstract

Objective – To investigate the reasons behind the cessation of 9 virtual (chat) reference services.

Design – Multiple case-study investigation

Setting – Academic, public, and special libraries in the United States.

Subjects – Five academic libraries, 1 public library, and 3 library consortia.

Methods – The initial method used to identify discontinued chat reference services was via a posting to DIG_REF, the digital reference services listserv. From this initial posting, librarians on the list also suggested other cases, for a total of 7. Two cases later came to the authors’ attention and were added. The 9 cases included academic libraries, a public library, and consortia who used tools provided by four major software vendors, giving a good cross-section of the types of libraries participating in VR and the major software packages available at the time. A contact person for each discontinued service was invited to participate. The participants were geographically scattered, so data were collected via e-mail questionnaires, telephone interviews, and examination of available documents describing the services, audiences, demographics, operations, and any decision-making processes either for starting or discontinuing the service. The authors then compared and contrasted results obtained from the 9 services. The authors analysed reasons for discontinuation of services, the decision making processes, volume of questions, service hours, marketing, evaluation, and whether the service might be restarted at some time in the future.

Main Results – Although the VR services discussed were being delivered by diverse organizations, some commonalities emerged. All 9 services used existing staff and added chat reference to their duties. In all but one case, the hours of service offered when VR began were within the hours that the libraries were already open to the public. In the exception (Case E), there were hours of service after the libraries were closed, from 7-10 p.m. These shifts were covered by librarians working from home.

Decisions to either begin or suspend services were generally made by small informal groups or committees. Some cases began with pilot studies and received positive responses. Other than these pilot studies, little pre-planning appeared to have been done. For example, there were no extensive advance surveys or studies to determine potential demand or markets for the service, or projected costs and benefits. Only one service had a strong, multi-stage evaluation plan. In future launches of virtual reference services, both extensive pre-planning and detailed measures for evaluation could be helpful in avoiding some problems.

Major reasons for discontinuation fell into 6 categories: funding problems, staffing problems, technical problems, institutional culture conflicts, low usage overall or low usage by target populations. A table (Table 1) was included that actually listed 7 categories of reasons for discontinuation, but the last one listed, “Software Change,” is discussed as part of “Technical Problems” in the text, though it is separated in the table (527). This is not immediately clear in the article, and thus the table is slightly confusing. Four cases reported funding problems as the major reason for discontinuation. Low volume or low volume for target population were primary reasons for discontinuation by 4 cases, and secondary reasons for discontinuation by 4 cases. If one combines primary and secondary categories in this table, low volume is the most frequently cited reason for discontinuation. Low volume was determined to be “driven by a complex combination of variables including marketing strategies, insufficient hours of operation, and [failure to provide] an ample amount of time for a service to gain momentum” (527-528).

Funding was cited as the primary reason for cancellation in 4 cases. Technical problems were listed as primary or secondary factors for suspension of service in 2 cases, and software change was the secondary factor behind the suspension of service in 2 cases; one service reported that up to one third of chat sessions were ‘lost” (527). In many cases, these sites were early adopters of VR, or even beta test sites, so technical issues are perhaps not surprising.

Staffing was the primary or secondary reason for discontinuation in a total of 3 cases. Staffing problems included insufficient questions to keep staff interested; however, concerns about extending service hours and thereby putting additional pressure on existing staff were also raised. Clashes in institutional cultures were reported as secondary reasons for discontinuation of the service in 2 consortia. It should be noted that the sample size is small; as the authors point out, the literature on unsuccessful or discontinued virtual reference services is quite limited.

Pop-up surveys were used by some cases to evaluate service success and user satisfaction. User satisfaction was reported as being high, but return rates for the surveys were very low, as can be expected since pop-ups are often blocked or ignored. The authors recommend alternative means of evaluation, such as focus groups and more detailed surveys. The authors also discuss alternative technologies and user expectations for services such as IM or SMS text messaging, and suggest evaluation of user expectations and desires in these areas.

One question which arose was what constituted success in chat reference – was it usage/volume, or some other measure, such as whether chat reference queries were answered correctly more frequently than reference questions asked in other ways, such as by telephone or in person. Other key concerns identified include the importance of strong marketing, service usability, and the importance in a consortial environment of having someone responsible on a full-time basis for directing and spearheading the project. Many of the suspended services reported that they could potentially become re-interested in VR service, as issues are resolved and the technologies available mature.

Finally, the authors provide detailed recommendations for single and consortial VR services, and offer directions for future research. Some suggestions for future research include ways to bridge the apparent gulf between expectations and technological preferences of users versus those of librarians and others staffing these services, as they often appear to have marked differences. Other areas of interest could include causes for low question volume, and possible relationships among various factors contributing to such poor usage along with an investigation of what makes successful services work well. Research on IM or SMS reference services as compared to those using other chat technologies and cost-effectiveness studies or evaluations based on goals and measured outcomes are also needed.

Conclusion – VR services have had erratic success rates and bumpy beginnings, but many libraries report continued interest The libraries in this study all indicated that they would consider, or are considering, re-starting such services. Issues of staffing, technological usability and reliability, institutional culture, funding, marketing, and volume are key. As services and technologies mature, interest in offering VR services in one form or another continues to grow.