Consumer Health Information Websites with High Visual Design Ratings Likely to Be also Highly Rated for Perceived Credibility

Abstract

A Review of:
Robins, D., Holmes, J., & Stansbury, M. (2010). Consumer health information on the web: The relationship of visual design and perceptions of credibility. Journal of the American Society for Information Science and Technology, 61(1), 13-19.

Objective – To answer two research questions: 1) What is the relationship between the visual design of a consumer health information web site and perceptions of the credibility of information found on it? 2) Is there a relationship between brand recognition, visual design preference, and credibility judgments?

Design – Qualitative (correlation of rating of response to stimulus) and quantitative (credibility coding of participant comments)

Setting – Not stated; assumed to be academic institutions in the United States.

Subjects – Thirty-four participants over the age of 35 (34 for statistical power and age over 35 on the hypothesis that this age group is most likely to seek health information on the Internet).

Methods – Screen shots of 31 consumer health information sites chosen from the results of a Google search using the term “consumer health information” were converted to slide format and shown to participants. The 31 sites included 12 of the top ranked consumer health information sites derived from three sources: the Consumer and Patient Health Information Section (CAPHIS) of the Medical Library Association (MLA), the MLA itself, and Consumer Reports. Participants were read and shown a script explaining the process prior to being asked to view and rate the 31 sites. Participants were first shown a blank slide with a crosshair to focus attention. Then a stimulus slide was shown for 2.8 seconds. A blank black screen was shown while they determined their rating. Participants were first asked to rate the visual design and aesthetics of the 31 web sites using a rating scale of -4 to -1 for negative judgments and +1 to +4 for positive judgments. Then they were asked to remember why they had made positive or negative ratings and why some web sites were preferred to others. The process was repeated with the slides re-ordered, and participants were asked to rate the credibility of the sites using the same rating scales. Upon completion, participants were asked to recall their reasons for positive or negative credibility ratings. All ratings were converted to positive numbers and a scale of 1-8 was used to present results. A variety of statistical analyses were carried out on the data, including correlation, means ratings, and rankings. In addition, all solicited comments on credibility were coded using Fogg’s four types of credibility (surface, earned, presumed, and reputed) in order to try to understand why participants rated the credibility of sites as they did.

Main Results – For the first question, concerning the relationship between visual design preferences and perceived credibility, the results are complicated. A statistically significant correlation was reported between visual design preference and perceived credibility in 8 of the 31 sites (26%). In these instances where visual design is rated highly, so is credibility. When visual design ratings were ranked highest to lowest, credibility ratings followed the same pattern. Similarly, when credibility ratings were ranked highest to lowest, visual, design ratings followed. A t-test confirmed that sites perceived to have higher credibility were also perceived to have better visual design. Furthermore, when design and credibility ratings were compared to site traffic rankings, as measured by Alexa (http://www.alexa.com), the trend was for both visual design and credibility ratings to decline as the site traffic ranking declined. This finding was also confirmed by a t-test. While there is not an exact relationship, the tendency is for sites with higher visual design ratings to also receive higher ratings for perceived credibility.

On the second question, concerning the relationship between brand recognition and visual design and perceived credibility judgments, the results suggest a possible influence of brand name. This relationship is not clear, and as visual designs were always presented and rated first, there is possibly a co-founder. The analysis of participant comments found that participants performed credibility judgments in a very short time using a variety of criteria, including visual design, source of the site, reputation of the site, and prior use. There were negative reactions to the use of advertisements, drug and insurance company sponsorship, and dot com sites, as well as some suspicion that non-US consumer health information sites were less trustworthy.

Conclusions – Visual design judgments bore a statistically significant similarity to credibility ratings. Sites with recognizable brands were highly rated for both credibility and visual design, but this relationship was not statistically significant. The relationship is complicated and more research is needed on what visual design cues are important to credibility judgments.