Evidence Summary

Pediatric Residents and Interns in an Italian Hospital Perform Improved Bibliographic Searches when Assisted by a Biomedical Librarian

A Review of:

Gardois, P., Calabrese, R., Colombi, N., Lingua, C., Longo, F., Villanacci, M., Miniero, R., & Piga, A. (2011). Effectiveness of bibliographic searches performed by paediatric residents and interns assisted by librarian. A randomised controlled trial. Health Information and Libraries Journal, 28(4), 273-284. doi: 10.1111/j.1471-1842.2011.00957.x

Reviewed by:

Mathew Stone
Assistant Librarian
Bradford Teaching Hospitals NHS Foundation Trust
Bradford, United Kingdom
Email: mathew.stone@bthft.nhs.uk

Received: 27 Nov. 2012 Accepted: 22 Jan. 2013

2013 Stone. This is an Open Access article distributed under the terms of the Creative Commons‐Attribution‐Noncommercial‐Share Alike License 2.5 Canada (http://creativecommons.org/licenses/by‐nc‐sa/2.5/ca/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one.

Abstract

Objective – To establish whether the assistance of an experienced biomedical librarian delivers an improvement in the searching of bibliographic databases as performed by medical residents and interns.

Design – Randomized controlled trial.

Setting – The pediatrics department of a large Italian teaching hospital.

Subjects – 18 pediatric residents and interns.

Methods – 23 residents and interns from the pediatrics department of a large Italian teaching hospital were invited to participate in this study, of which 18 agreed. Subjects were then randomized into two groups and asked to spend between 30 and 90 minutes searching bibliographic databases for evidence to answer a real-life clinical question which was randomly allocated to them. Each member of the intervention group was provided with an experienced biomedical librarian to provide assistance throughout the search session. The control group received no assistance. The outcome of the search was then measured using an assessment tool adapted for the purpose of this study from the Fresno test of competence in evidence based medicine. This adapted assessment tool rated the “global success” of the search and included criteria such as appropriate question formulation, number of PICO terms translated into search terms, use of Boolean logic, use of subject headings, use of filters, use of limits, and the percentage of citations retrieved that matched a gold standard set of citations found in a prior search by two librarians (who were not involved in assisting the subjects) together with an expert clinician.

Main Results – The intervention group scored a median average of 73.6 points out of a possible 100, compared with the control group which scored 50.4. The difference of 23.2 points in favour of the librarian assisted group was a statistically significant result (p value = 0.013) with a 95% confidence interval of between 4.8 and 33.2.

Conclusion – This study presents credible evidence that assistance provided by an experienced biomedical librarian improves the quality of the bibliographic database searches performed by residents and interns using real-life clinical scenarios.

Commentary

Searching for evidence is a core activity in the practice of evidence based medicine and hence there is an obvious opportunity for librarians, as expert searchers, to play a major supporting role. Although previous studies have already examined a range of library based interventions aimed at improving search performance for health evidence, the lack of reliable research for one-to-one librarian support for healthcare staff is the justification for this study. Indeed the authors voice a more general discontent with the quality of research in this area by highlighting the lack of experimental study designs employed and the subjective manner in which outcomes have often been measured.

The intervention tested in this study is one tailored more for one-off evidence based medicine projects such as devising local guidelines or commissioning new services, rather than as an adjunct to routine patient care where clinicians will presumably struggle to find the time to search bibliographic databases (with or without the support of a librarian) between patient appointments.

The use of an assessment tool (albeit one that appears to have not been validated) to measure the quality of the search process is a main strength of this study. This combined with the random allocation of subjects to intervention/control groups (which reduces the impact of confounding variables) elevates the study to a very high level of evidence. The use of an objective assessment tool minimizes the effects of observer bias, a factor which has potentially distorted studies of similar interventions where subjects have been asked to assess their own search skills.

The sample size of this study was small (only 18 participants), but it was adequately powered to produce a statistically significant result due to the large difference between the outcomes in the two groups. However, while we can be confident that librarian assistance causes an improved search performance, we can be less sure about the magnitude of this improvement. The wide 95% confidence interval signifies that we can only be confident that the extent of improvement lies somewhere between 4.8 and 33.2 points. This means librarians will have to ask themselves if they can justify providing this kind of one-to-one support if the true impact of this assistance is at the lower end of this confidence interval. A larger sample size would have produced a narrower confidence interval and therefore allowed us to assess the effect size of this intervention with greater precision.

The generalizability of these results to the wider population of medical residents and interns is strengthened by the high participation rate in this study. 78% of invitees were recruited in to the study and so the possibility of a non-response bias (where only confident searchers agree to participate) resulting in an unreflective sample is minimized. However, the degree to which we can generalize from interns and residents to a more general clinical population is uncertain. For instance, one might expect interns and residents who are not long out of medical school to be comparatively better searchers of bibliographic databases than their more experienced clinical colleagues and consequently less likely to benefit from the intervention. But despite the possibility of a selection bias, the effect of which would most probably be to moderate the positive result of this intervention, this randomized controlled trial provides reliable evidence that bibliographic databases searches are improved when clinicians are supported by a librarian.

Librarians can take a couple of key points from this study, one quite specific and the other more general. Sub-category analysis reveals that the particular component of the intervention that was most beneficial to clinicians was the help translating PICO terms into search terms, so librarians would be well advised to focus their efforts in this direction. More generally, librarians now have compelling evidence showing how they can improve the evidence based medicine process by helping the clinician to search the literature more effectively. Librarians can therefore push confidently (if they are not doing so already) for greater involvement in evidence based medicine projects within their organizations.