User Engagement with Mental Health Videos on YouTube

Introduction: Mental health is a primary determinant of well-being, and as more people look online for mental health information, YouTube is an increasingly important information source. Although authoritative organizations such as the World Health Organization post videos to YouTube, when retrieved these videos are interspersed with personal, commercial, governmental, television or other media segments, and institutional videos. YouTube was searched for videos on mental health to measure user engagement with these videos. It was hypothesized that videos posted to YouTube that contained personal narratives would generate more user engagement in terms of more video view counts, likes, and number of comments. Methods: YouTube was searched for mental health information using three different search terms and phrases: “depression,” “bipolar disorder,” and “mental health.” The first 20 results for the terms depression and bipolar disorder were screen captured and for the search phrase mental health the first 40 videos were screen captured. All 80 videos were categorized according to video producer type and analyzed using YouTube metrics including number of “likes,” view counts, and comments to measure user engagement with the videos. Results: The majority of videos returned in the top results were posted by laypersons and the videos focus on the poster's personal experience (38%) followed by videos produced for television and other media (29%). Videos that contain personal narratives and experiential knowledge generate the most user engagement and are preferred sources for users searching for mental health information. Discussion: Users’ greater engagement with personal videos indicates that there is an important role for librarians and information professionals in assisting users in deciding what mental health information is accurate, authoritative, and reliable regardless of the authority of the video producer. In addition, the results of this research might inform best practices for professional organizations posting videos to YouTube.


Introduction
Research shows that mental illness is the second leading cause of disability and premature death in Canada, that the Canadian economy loses $51 billion a year to mental illness in terms of lost productivity and health care costs, and one in five Canadians will experience a mental illness during their lifetime but only one-third of those who need mental health services receive them [1]. According to the Canadian Mental Health Association, mental illnesses include anxiety disorders, attention deficit disorder, depression, bipolar disorder, eating and mood disorders, suicide, violence, psychosis, schizophrenia, and self-injury [2]. People coping with or managing a mental illness often turn to online sources for support and health information. Increasingly, this includes social media web sites that offer peer-to-peer, expert, and user-generated content such as YouTube.
Patients rely on the Internet more frequently than their physicians as a source of health care information [3]. For example, 81% of adult Americans report having researched at least one specific health topic online, and 21% of those health seekers sought online information about depression, anxiety, stress, or mental health issues [4,5]. Increasingly, social media web sites such as YouTube play an important role in online health searches [5,6]. Participation in social media is significantly correlated to increased coping skills and positive health outcomes [7,8]. Although access to social media sites that offer peer-to-peer support and personal narratives (experiential knowledge) is important for those coping with mental health issues, user-generated content, encouraged by sites such as YouTube, is increasingly supplementing and, at times supplanting, expert medical knowledge.
YouTube is widely regarded as the most important source of visual entertainment and information in the world with more than 1 billion unique users visiting YouTube each month and 72 hours of video uploaded to YouTube every minute. In 2011, YouTube had more than 1 trillion views or around 140 views for every person on Earth. The use of YouTube is a political, social, and economic force that is changing the way in which people Tami Oliphant. 2 School of Library and Information Studies, 3-04 Rutherford South, University of Alberta, Edmonton, AB, T6G 2J4. 1 This paper is peer reviewed. 2 Corresponding author. (e-mail: toliphan@ualberta.ca). connect, access, share, and exchange information. However, YouTube is a complex information environment. It is a site from which videos containing erroneous and misleading information are as likely to be retrieved from a search string for mental health as videos produced by the World Health Organization.
Further complicating YouTube's information environment is the lack of transparency regarding the ordering of search results. In October 2012, YouTube announced that ''We've started adjusting the ranking of videos in YouTube search to reward engaging videos that keep viewers watching.'' [9]. YouTube's default relevancy sorting now retrieves and orders video results based upon watch time of videos rather than number of views in an effort to increase video viewership and ad revenue. The search results retrieved from YouTube's altered search algorithm may have implications for both video producers and users who rely on YouTube as a source for mental health information, because total watch time does not necessarily correlate to information that is relevant, authoritative, accurate, and reliable. Furthermore, in addition to this policy of ranking results according to watch time, there is an utter lack of transparency and consistency in how YouTube's search algorithm determines relevant search results.
Understanding how users engage with mental health YouTube videos as a health information source is increasingly important for information and healthcare professionals. User engagement is a complex construct that involves user feelings, mental states, and interactions. Engagement has been problematized in the literature as a ''user's response to an interaction that gains, maintains, and encourages their attention, particularly when they are intrinsically motivated'' [10]. O'Brien and Toms expanded upon this definition and suggested that engagement is ''a quality of user experiences with technology that is characterized by challenge, aesthetic and sensory appeal, feedback, novelty, interactivity, perceived control and time, awareness, motivation, interest, and affect'' [11]. One way of measuring user engagement is by analyzing YouTube's metrics. In 2009, YouTube added simple features to measure user engagement with YouTube content such as likes, dislikes, comments, view counts, and more recently, watch time. For the purposes of this study, user engagement was operationalized as attention, interactivity, and feedback to mental health videos as measured by YouTube's metrics * number of comments, likes, and number of view counts.
An emerging body of research examines the range and accuracy of health related videos uploaded to YouTube [12Á16]. Results from these studies show great variability in the quality, accuracy, and purpose of health information found on YouTube. For example, Singh et al. [16] found that of 102 relevant videos about rheumatoid arthritis, 54.9% were classified as useful and 30.4% were deemed misleading. All videos uploaded by university channels and professional organizations included in the sample provided useful information but formed only 12.7% of total videos, whereas 73.9% of medical advertisement videos and videos by for-profit organizations were misleading. Similarly, Keelan et al. [17] evaluated 153 YouTube videos about immunization. They measured users' engagement with the videos using view counts and viewer reviews indicated by the star-rating system. The videos were then rated for scientific accuracy based upon whether they were substantiated or unsubstantiated by the 2006 Canadian Immunization Guide. None of the positive videos contradicted the Guide but nearly half of the negative videos (22 of 49; 45%) did. Although researchers found that almost half of the videos were positive and only 20% were negative, when compared with positive videos negative videos were more likely to receive a rating, they had a higher mean star rating, and they had more views. Keelan et al. [17] found that among the positive videos, public service announcements (PSAs) from governmental agencies or nongovernmental organizations received the lowest mean ratings and the fewest views.
Briones et al. [18] found that the majority of the 172 YouTube videos about the human papillomavirus (HPV) vaccine were negative in tone, and similar to the findings of Keelan et al. [17], the negative videos were liked more often by the viewers than positive or ambiguous ones. A common call by these researchers is for public health and medical professionals to be cognizant of the nature of available health related information to be better equipped to deal with patients who acquire information from YouTube and other Internet sources [12Á14, 16].
These studies show that YouTube videos produced by experts or organizations (whether governmental or nongovernmental) are the least viewed and generate far less interaction and comments among users than videos produced by for-profit companies or laypersons [12, 13, 15Á19]. Paek et al. [19] found that PSAs about child abuse prevention posted on YouTube that were produced by experts or organizations were not as effective in enhancing attitudes towards the PSA and issue importance as those videos produced by a perceivably similar group or layperson. Similarly, Pant et al. [15] found that YouTube videos on myocardial infarction produced by lay individuals generated the most interaction with users through likes and dislikes in addition to generating more comments than videos produced by experts. Other research demonstrates that YouTube users engage with personal narratives more deeply than informational videos [20,21].
These findings have two implications. First, governments and nonprofit organizations would be wise to encourage individual participation if they wish to increase user interaction and engagement with video content on sites such as YouTube. Second, these studies indicate that for public health authorities and medical professionals attempting to communicate health information via YouTube packaging is as important as content.

Research questions
Despite more people searching online for health information, there is a dearth in the literature investigating user engagement with mental health videos on YouTube as measured by YouTube metrics. Although highly regarded professional organizations such as the Canadian Mental Health Association and the United States National Institute of Mental Health produce videos about mental health, these videos are interspersed with personal, institu-tional, and commercial videos when searched for and retrieved on YouTube. This research project explores user engagement with mental health videos on YouTube. Because mental health disorders are often related to issues of identity and narrative construction, it was hypothesized that videos in which the producer draws upon personal narrative and experiential knowledge will generate more engagement among users regardless of the authority of the video producer.

Methods
YouTube was searched for mental health information using three different search terms and phrases: ''depression,'' ''mental health,'' and ''bipolar disorder.'' The first 20 relevant results returned for the terms depression and bipolar disorder were screen captured and analyzed. For the phrase mental health the first 40 relevant videos were screen captured and analyzed for a total of 80 videos. The bipolar disorder search retrieved about 663,000 results, the depression search retrieved about 2,440,000 results, and the mental health search retrieved about 4,850,500 results. Because so many more videos were retrieved from the search term mental health compared with depression and bipolar disorder the number of videos analyzed for this search phrase was doubled.
Each video was categorized according to producer type (i.e., personal, commercial, organizations and (or) government, television and (or) media, and university) and then analyzed using YouTube metrics (i.e., source type (producer), likes, view counts, and number of comments). Analyzing YouTube metrics as a means of measuring user engagement is a method commonly used in research [15,18,22]. Videos retrieved in the top results were assessed for relevance. Occasionally, music videos or videos were retrieved in a language other than English. Because YouTube search results are unstable in that the same search carried out mere minutes after the original search can yield different results, each dataset was collected in a single session on 15 May 2013 (depression search), 16 May 2013 (mental health search), and 18 May 2013 (bipolar search) to provide a snapshot of YouTube search results for analysis. This data collection method is a potential limitation to the study because YouTube search results are not stable and thus the ordering of videos captures search results from a specific time. It is highly unlikely that the same search results, in the same order, would be retrieved from YouTube by a subsequent search. However, the mix of video producers incorporated into search results retrieved from YouTube is a consistent feature of YouTube.

Findings
Videos were categorized according to video producer type (Table 1). Using a content analytic approach, video producer type categories are mutually exclusive. For example, the personal category included videos produced and posted by lay individuals and it also included videos in which celebrities were interviewed about their experiences with mental health disorders. Videos featuring the personal experiences of celebrities were categorized as personal rather than television or media. Conversely, if the video was a mental health media segment and featured a celebrity as an example of a famous person with the disorder, the video was categorized as television or media.
The majority of videos returned in the top 20 (depression and bipolar disorder) and top 40 results (mental health) were personal in nature (37.5%), followed by television or media-related (28.75%), commercial (13.75%), university (12.5%), and organizations or governments (7.5%) ( Table 2). The bipolar disorder search retrieved just one video produced by an organization, a In terms of user engagement as measured by total view counts, personal videos were viewed at over twice the rate of commercial videos (3,727,622 vs. 1,653,558) and roughly seven times the views of government-or organizationproduced videos (502,944 views). These data support previous study findings that people value experiential knowledge and they engage with videos produced by people like themselves. However, for the mental health search, personal videos ranked third for total view counts behind commercial and organization and (or) government produced videos. These findings indicate that users engage with personal videos particularly when they are focused on a mental health issue such as depression or bipolar disorder rather than videos that broadly discuss mental health. In the YouTube search results the lowest number of total video view counts came from the television or media category (455,671 total views) despite being the second most frequently returned video producer type (28.75% of all videos retrieved). The results showed that users are watching videos produced by television or other media about bipolar disorder more frequently than commercial videos and at nearly the same rate as they watch personal videos.
Furthermore, 40 videos were sampled for the mental health search compared with just 20 for the depression and bipolar search. Despite being twice as large as the other searches, users watched fewer videos about mental health. As noted previously, users were not engaging with personal mental health videos as often as they were engaging with commercial and organizations and (or) government mental health videos. Users watched commercial videos produced by an organization called psychetruth, Healthguru's ''5 Types of Bipolar Disorder'' and videos from organizations and government such as World Health Organization's ''WHO: Mental Health,'' for example, more frequently than personal videos. Many health organizations have a mandate to promote mental health and commercial organizations advertise their products and services under the guise of promoting mental health, which may partially account for the higher total number of views. Table 3 shows the average number of views per like and per comment for each video producer type. Thus, lower numbers indicate greater user engagement with the video. For example, the depression search indicates that for personal videos users liked the video for every 64 views on average, and users commented on the video for every 122 views on average. Videos in which comments and likes were disabled were excluded from these calculations. These numbers were aggregated and consequently, some individual videos in the same producer type category had greater or lesser amounts of interaction compared with others. As hypothesized, personal videos generated more user engagement as measured by number of likes and number of comments across all three searches than other video producer types with the sole exception of number of views per comment for the mental health search. Videos produced by commercial entities generated a fair amount of user engagement in both likes and comments. For those videos produced by organizations and (or) governments, television or media, and universities, users engaged with the video far more frequently in terms of likes but not comments. This finding indicates that these video producer types are not perceived by users as being as interactive in the same way personal videos are. Similarly, the number of likes and comments (expressed as a ratio number of view counts / number of likes and comments) was the highest for governmental videos with the sole exception of the average number of views of television or other media produced videos per comment for the bipolar disorder search, which averaged 1,487 views per comment.

Discussion
There are a number of implications of these findings for librarians and information professionals. As hypothesized, users engaged more frequently with videos that are personal in nature as measured by total view counts and YouTube viewers tend to comment the least for videos produced by organizations, television or media, and universities, whereas they comment most frequently on videos produced by commercial organizations and those that are personal in nature. However, in terms of likes, the commercial videos were the least liked in all three searches. Users perceive personal videos to be more engaging in terms of interactivity, feedback, and attention as measured using YouTube's metrics. Although YouTube generally is a medium that facilitates user engagement, merely posting a video to YouTube does not guarantee that users will respond, interact, or engage with it. Searches for the phrase mental health have very low view counts and user interactions regardless of video producer. This suggests that users are not using the search string mental health as frequently as depression or bipolar disorder when searching for videos. For example, the personal video ''Mental Health Awareness Week 2013'' was returned in the 15th position of results but did not have a single view, like, or comment.
YouTube retrieves a wide array of videos produced by individuals, governments, commercial organizations, television or media, and universities, but an unexpected finding was the randomness in which YouTube orders search results. Search results were not determined by view counts, number of comments, video producer, or date posted. Despite YouTube's October 2012 announcement that video results will be retrieved according to watch time, there is no way to ascertain if, and how, YouTube's search algorithm determines relevant search results.
Consequently, librarians can support users looking for online mental health or other consumer health information by drawing upon both expert sources and sources of information where experience is emphasized. In addition, the results of this research might inform best practices for professional organizations posting videos to YouTube. For example, including a personal narrative or story is most likely to generate more user engagement.
Future areas of research include analyzing the comments section of videos to determine what kind of information exchange, if any, occurs on the comments and to determine how positive or negative the comments are, and how accurate or authoritative the comments are. Another avenue of future research includes analyzing how YouTube determines the order of search results. The primary limitations of this study are suggested as future areas of research using a larger sample size, additional analysis of user comments, and further investigation of how YouTube renders search results. As more people search online and use video sites such as YouTube, it is crucial that library and information professionals understand how people are using, and engaging with, video to supplement their health-related information seeking practices.