Clinical Informatics Consult Service Positively Affects Some Clinical Decisions in the ICU

Abstract

A Review of:
Mulvaney, Shelagh A., Leonard Bickman, Nunzia B. Giuse, Warren E. Lambert, Nila A. Sathe, and Rebecca N. Jerome." A Randomized Effectiveness Trial of a Clinical Informatics Consult Service: Impact on Evidence-based Decision-making and Knowledge Implementation." Journal of the American Medical Informatics Association 15.2 (2008): 203-11.

Objective – To determine whether the provision of synthesized research evidence provided by the Clinical Informatics Consult Service (CICS) affects the clinical decision-making of clinicians working in intensive care units (ICUs).

Design – Non-blinded randomized control effectiveness trial.
Setting – ICUs in United States-based 658 bed university hospital providing tertiary care for adults and children.

Subjects – Clinical staff working within one of four ICUs who submitted a request for clinical information during the study period.

Methods – Valid requests submitted by clinical staff from the four clinical ICUs (medical, paediatric, trauma, or neonatal) were randomly allocated to receive information from the CICS (CICS provided) or no information (no CICS provided). Pre-consult forms, completed at the time of the request, examined reasons for the request and the clinical actions clinicians thought might be influenced by the search results. Requestors could opt out of the no CICS provided group either before or after the randomization of their request.

Responses to requests, supplied within 0.5 to 7 days as agreed with the requestor, included a search strategy and bibliographic references, a targeted list of full-text articles, and a written synthesis and critique of the relevant research.

Clinicians within both groups were free to conduct their own searches and reviews.

An online evaluation form, emailed to recipients, was used to assess the impact of the information supplied.
The evaluation form asked clinicians to record the time spent on their own searches, sources of information consulted including colleagues, the immediate and future impact of the information provided (either from the CICS or their own searches), what influence the information had on their clinical actions, whether there were any barriers to using the information, and quality and overall satisfaction with the results provided by the CICS.

Data was analyzed according to the randomized group assignment using standard intention-to-treat analysis for the main outcomes between the two groups. Statistical adjustments were made to control for possible clustering of responses or multiple ratings from individual clinicians.

The data was also analyzed on an efficacy basis depending on who provided the search results. The groups were Clinician only, CICS librarian-only, or Both Clinician and CICS librarian. Results from the Clinician only search group were used as a comparison to the remaining groups. This assessment did not take account of the randomization and therefore constitutes a cohort analysis.

Results were analysed by one of two methods using statistical software SAS Proc Mixed (v9) for multi-level quantitative data analysis, e.g., analysis of variance, and SPSS (v14) for all other quantitative data analysis, including descriptive statistics.

Main Results – The study period was conducted over 19 months: August 2004 to March 2006. During this time, 299 valid requests were received and 226 post consult evaluation forms were returned giving a response rate of 76%. Post consult forms were returned for 108/146 of the CICS provided group and 118/153 of the no CICS provided group. The 24% of requests that had no post consult evaluation were excluded from further analysis. Statistical tests, conducted to check for potential bias relating to missing data, suggested that the missing data had little impact on the findings.

Medical and neonatal ICUs accounted for the majority of completed forms (40.3% and 38.1% respectively). The majority of opt-outs (10.2% overall) were from the medical ICU. No significant difference in outcome variables was found between opt-out and other requests when tested using analysis of variance (ANOVA).

Evaluation forms were completed by 89 unique clinicians and over half (49) submitted more than one request. The average requests per clinician was 2.96, SD 3.17, range 1-15, and the average number of requests per clinician who submitted more than one request was 4.57, SD 3.55, range 2-15.

Total number of responses, mean, standard deviation, and Cohen’s d effect size were reported for the outcome variables based on intention to treat analysis. Results showed no significant difference between the groups on the immediate impact of the information provided, the number of articles read or the frequency with which clinicians consulted colleagues, either formally or informally. The potential future impact of the information was rated higher in the CICS group (p=<0.01) and clinician’s satisfaction levels in the CICS group were also significantly higher (p=<0.001). Only the specific action “different or new treatment” option showed a statistically significant difference between the CICS provided and no CICS provided groups. A significantly greater percentage of clinicians in the CICS provided group reported conducting their own searches (70.2%) compared to the no CICS provided group (36.8%).

Where clinicians reported that the information provided, either from their own searches or CICS, had no impact on patient care decisions, the reasons most commonly cited were that they were already doing what was recommended, there was insufficient evidence to guide their decision, or that the available evidence did not apply to their patient.

Analysis of the results on the basis of the intervention actually received showed both potential future impact and satisfaction were significantly higher for both the librarian-only (p<0.001 for both outcomes) and librarian plus clinician search groups (p<0.001 for both outcomes) compared to the clinician only group. In addition, there was a statistically significant difference (p=0.02) of future impact in favour of the librarian plus clinician search group over the librarian-only group. Time taken to complete searches also showed a statistically significant difference between the groups, which is not surprising given that time reported by the librarians included time spent searching and summarizing results, whilst the clinicians recorded time spent searching only.

Conclusion – Clinicians reported a higher level of satisfaction with search results provided by the CICS and rated the future impact of the information more highly. The CICS showed a statistically significant impact on some aspects of clinical decision-making, particularly with regard to treatment decisions. Provision of information by the CICS also appeared to encourage clinicians to undertake their own searches. However, the reasons for this are unclear.