Knowing How Good Our Searches Are: An Approach Derived from Search Filter Development Methodology

Abstract

Abstract

Objective – Effective literature searching is of paramount importance in supporting evidence based practice, research, and policy. Missed references can have adverse effects on outcomes. This paper reports on the development and evaluation of an online learning resource, designed for librarians and other interested searchers, presenting an evidence based approach to enhancing and testing literature searches.

Methods – We developed and evaluated the set of free online learning modules for librarians called Smart Searching, suggesting the use of techniques derived from search filter development undertaken by the CareSearch Palliative Care Knowledge Network and its associated project Flinders Filters. The searching module content has been informed by the processes and principles used in search filter development. The self-paced modules are intended to help librarians and other interested searchers test the effectiveness of their literature searches, provide evidence of search performance that can be used to improve searches, as well as to evaluate and promote searching expertise. Each module covers one of four techniques, or core principles, employed in search filter development: (1) collaboration with subject experts; (2) use of a reference sample set; (3) term identification through frequency analysis; and (4) iterative testing. Evaluation of the resource comprised ongoing monitoring of web analytics to determine factors such as numbers of
users and geographic origin; a user survey conducted online elicited qualitative information about the usefulness of the resource.

Results – The resource was launched in May 2014. Web analytics show over 6,000 unique users from 101 countries (at 9 August 2015). Responses to the survey (n=50) indicated that 80% would recommend the resource to a colleague.

Conclusions – An evidence based approach to searching, derived from search filter development methodology, has been shown to have value as an online learning resource. More information is needed about the reasons why people are using the resource beyond what could be ascertained by the survey results.