Web-Based Portal for Impact Evaluation Reveals Information Needs for Museums, Libraries and Archives

Abstract

Objective – This study reports on research into the information and support needs of practitioners in the museum, archive, and library sectors, who are undergoing an impact evaluation.

Design – Qualitative survey.

Setting – Web-based questionnaire.

Subjects – Twenty-one practitioners in the fields of museums, archives, and libraries.

Methods – The study made use of a small-scale web portal that provides impact evaluation research findings, toolkits, and examples of methods. The portal’s intent was to present to the users multiple views of the available information in order to overcome the problem of users not being able to identify their needs. A purposive sample group consisting of 50 practitioners from the museum, library, and archive fields was invited to participate in a questionnaire evaluating the website.

Main Results – Despite a fairly low response rate (49%) and poor distribution among the three sectors (museums, libraries, and archives), the results indicated a significant difference in the levels of knowledge and understanding of impact evaluation. Over half of the organizations surveyed had done some assessment of their institution’s economic impact, and there appears to be a rising trend towards doing impact studies for specific projects and developments. Nearly a quarter of the organizations had not undertaken any impact evaluation study previously. Practitioners already familiar with impact evaluation tended to look at broader range of fields for expertise, whereas those with less familiarity remained within their own sector. Practitioners with less experience preferred tools, guidance, and examples of methodologies as opposed to actual evidence of impact. The results also provided the authors with feedback on their web portal and how to organize the information therein.

Conclusions – One of the findings of the study was that the overall reaction to impact evaluation support through research evidence, guidance, and other mechanisms was positive. For most practitioners, evaluation itself and the level of understanding of impact evaluation are at early stages. The primary goals for those undertaking impact evaluation were found to be professional and organizational learning, thus there is a need for practical help and guidance in these areas. Time limitation appeared to be a significant factor in the responses – particularly with smaller organizations – suggesting that their portal material would provide much-needed assistance to such organizations. Finally, it was concluded that future emphasis should be placed on developing practical applications rather than pure research.