Introduction
Social media platforms are a popular means of sharing medical information online. TikTok in particular is a relatively new social media platform that has seen rapid adoption by adolescents and young adults, becoming the most downloaded social media application in 2020 with more than 1 billion monthly active users.
1,2 In particular, the popularity of the platform appears to have contributed to increased awareness of attention-deficit/hyperactivity disorder (ADHD), with some individuals seeking a diagnosis after watching videos about ADHD on the platform.
3,4 The hashtag “#adhd” is currently the seventh most popular health-related hashtag on the platform.
2 Although social media can reduce mental health stigma and improve health literacy, there is also concern about misinformation and the potential for illness/health anxiety (“cyberchondria”) due to the volume of unmoderated, user-generated content online.
2,5,6 For example, TikTok videos have been implicated in a recent rise of tic-like behaviours in adolescents, and it is thought that exposure to tic-related videos is responsible for this phenomenon.
7-9
Additionally, social media platforms use proprietary algorithms that focus on increasing user engagement and may promote videos that do not necessarily reflect accurate health information. A recent systematic review found that the prevalence of health misinformation was high across almost all social media platforms.
6 With TikTok being a relatively new social media platform, there is little research on the quality and accuracy of its content. As a result, there have been increasing calls for more research to understand the potential implications of TikTok on health information dissemination.
2 Recent studies examining TikTok videos have found that misinformation was common for a variety of medical topics including those about acne, diabetes, and cosmetic surgery.
10-12 To our knowledge, no study has examined the quality of mental health information disseminated on TikTok. Thus, our objective was to assess the quality of medical information in the most popular TikTok videos related to ADHD.
Methods
The TikTok mobile phone application was queried with a search for the hashtag “#adhd” on July 18, 2021 (
Figure 1). The proprietary search algorithm returns the most popular videos based on the number of views and likes per video. Our inclusion criteria were videos that specifically described or educated viewers about: ADHD symptoms/diagnosis, lived experience with ADHD, or ADHD management. Exclusion criteria were videos with no audio or text, non-English videos, videos unrelated to ADHD, or duplicate videos. Videos were sorted in descending order by view count, and the inclusion criteria were applied until 100 videos were identified. The sample size of 100 videos was chosen as a feasible and representative sample size to capture the most viewed videos on the platform. Previous studies have used similar sample sizes to analyze health-related videos on social media platforms such as TikTok and YouTube.
10,13-15 Videos meeting the inclusion criteria were then included for extraction of video characteristics and further analysis.
One psychiatrist and one psychiatry resident (AY, EN) with clinical experience in the diagnosis and management of ADHD independently assessed the videos using a categorical classification often used to assess the content quality of social media videos.
16,17 Videos were viewed, analyzed, and classified as: (1) useful; (2) personal experience; or (3) misleading. Videos were classified as useful if the video contained scientifically correct information about any aspect of ADHD: prevention, symptoms, diagnosis, treatments, or other information (e.g., etiology, psychopathology, epidemiology, prognosis). Videos were classified as personal experiences if it described a user's own personal or anecdotal experience of ADHD symptoms or treatment. Videos were classified as misleading if it contained information lacking scientific evidence (e.g., unsubstantiated claims about treating ADHD). If a personal experience video additionally contained any generalized misleading statements, it was classified as misleading rather than personal experience.
Where applicable, raters referenced the Canadian ADHD Resource Alliance (CADDRA) Canadian ADHD Practice Guidelines, Fourth Edition, and American Academy of Pediatrics ADHD Clinical Practice Guidelines.
18,19 In cases where there was disagreement between the first two raters, a third psychiatrist rater (EAJ) independently rated the video to decide on the final rating.
All videos were also assessed using the Patient Education Materials Assessment Tool for Audiovisual Materials (PEMAT-A/V), a validated instrument to assess the understandability and actionability of audiovisual patient education videos.
20 Finally, videos were also rated on the
Journal of American Medical Association (JAMA) benchmark criteria,
21 a four-point scale which assesses medical information quality based on: (1) authorship (all authors and contributors, their affiliations, and relevant credentials should be provided), (2) attribution (references and sources for all content should be listed clearly, and all relevant copyright information noted), (3) disclosure (any sponsorship, advertising, underwriting, commercial funding arrangements or support, or potential conflicts of interest disclosed), and (4) currency (dates that content was posted and updated are indicated).
Video metadata was extracted automatically using an open-source web scraper using TypeScript programming language (TikTok-scraper). Descriptive characteristics for each video included: user type (individual, organization, or healthcare provider [HCP]), number of views, number of likes, and number of shares. The kappa statistic was calculated to assess inter-rater reliability for the initial video classification between the two raters. Student's t test was used to compare video characteristics, PEMAT scores, and JAMA scores between HCPs and non-HCPs video uploaders. Pearson's chi-square test was used to compare frequencies of the video classifications between HCPs and non-HCPs. One-way analysis of variance was used to compare video characteristics, PEMAT scores, and JAMA scores between the three video classifications. A P-value of less than 0.05 was considered significant for all analyses. All statistical analyses were performed using Stata 17.0.
Discussion
In this analysis of popular TikTok videos about ADHD, there were over 2.8 million views per video and each video was shared on average 31,000 times. Approximately half of the videos analyzed (52%) were misleading, with non-HCPs uploading most of these videos (49 out of 52 videos). There was substantial agreement among raters on the classification of the videos as useful, misleading, or personal experience. Our study is the first to show that misleading videos about ADHD are being widely disseminated and viewed on TikTok.
These findings are in line with previous studies of TikTok videos that have found high rates of misinformation for medical conditions, including acne and diabetes.
10,11 Our results are also similar to a study by Thapa et al. that analyzed YouTube videos about ADHD and found that 38% of analyzed videos were misleading and only 5% were useful.
22 The study also found that first-person or personal experience videos received the highest engagement (i.e., likes and views), while misleading videos were the most common. Our study replicated these findings on TikTok: personal experience videos received the highest engagement (i.e., views, likes, and shares), while misleading videos were the most common. These consistent findings across different social media platforms suggest that viewers are most drawn to videos made by individuals with lived experience, and less so towards institutional or HCP-created videos. Increasingly, social media influencers and independent video creators have been found to have an outsized influence on the dissemination of health information online.
23-25 Indeed, in our study, none of the top 100 videos were uploaded by corporations, health organizations, for-profit entities, or non-profit entities, and only 11% of videos were uploaded by HCPs. This limited presence of HCPs on TikTok has been observed in other studies as well.
10 Overall, HCPs did score significantly higher on the JAMA benchmark criteria and made significantly more useful videos and less misleading videos. Although this suggests that healthcare providers do upload higher quality content on the platform, 27% of HCP videos (3 out of 11) were still rated as misleading.
We additionally found that all videos about ADHD were highly understandable, scoring over 90% on the PEMAT-A/V understandability score. The understandability score does not reflect accuracy, but merely that the information is presented in an understandable manner. Thus, individuals may be seeing videos about ADHD on the platform that are highly understandable and yet misleading. This was seen in our qualitative review of misleading videos, which often had an oversimplified or reductionist explanation of ADHD.
The findings from our study provide several important insights into the dissemination of medical information about ADHD on TikTok. Approximately half of all videos analyzed were misleading, and the misinformation they contain has the potential to contribute to health anxiety or lead to increased healthcare utilization.
26 The proprietary TikTok algorithm has been found to have a propensity to show users similar videos over time, which may further propagate misleading videos.
27,28 Additionally, viral trends on TikTok,
29 “echo chambers” in social media,
30 and romanticization of mental health symptoms
9,31 may perpetuate the dissemination of misleading information on TikTok. Increased healthcare use may occur for individuals who seek clinical attention after viewing TikTok videos about ADHD, a phenomenon that has been widely documented by major news organizations.
3,4 Since self-report of ADHD symptoms may be over-endorsed,
32 there may be an increased risk for overdiagnosis or misdiagnosis in these individuals as well. In general, there has also been renewed interest over rising rates of ADHD diagnoses,
33-35 and debate about the overdiagnosis of ADHD.
35
TikTok has become particularly popular over the course of the COVID-19 pandemic. The pandemic context raises the question of whether individuals may be misattributing difficulties from pandemic public health measures (e.g., having difficulty with maintaining attention when viewing a screen in isolation for prolonged periods) to ADHD symptoms.
36,37 One related example is the rapid increase in tic-like behaviours that have been observed over the course of the COVID-19 pandemic.
7,9 Tic disorder specialists have seen a dramatic increase in referrals for tic-like behaviours in primarily adolescent females that are inconsistent with usual presentations of primary tic disorders. These presentations are thought to be most consistent with a diagnosis of functional neurological disorder, related to stressors over the course of the pandemic. The sociogenic propagation of tic-like behaviours across TikTok is a reflection of the impact of social media platforms on health behaviours.
7,9
Limitations of our study include TikTok's proprietary search algorithm results, which exclude video ads and do not allow for systematic searching of all videos or deleted videos. Not all videos contained detailed information to identify whether an uploader was a health professional or not. Additionally, the current study only screened for the top 100 most popular videos. Less popular videos may have different content quality and characteristics not captured in this study. Also, the conceptualization of what is considered as a misleading, useful, or personal experience video was done from the perspective of raters who are HCPs. This perspective may not necessarily be in line with healthcare consumers or patients. For instance, while some videos may not be useful from the perspective of an HCP, others may find it useful for other reasons. Additionally, although our inter-rater agreement was high, videos classified as useful could potentially be classified as personal experience videos. Videos on the platform may also be recorded to be humorous or spontaneous, and not necessarily recorded with the intent to disseminate medical information. However, even videos made without intent to disseminate medical information may describe non-specific symptoms, overgeneralizations, and characterizations about ADHD that could be misleading to viewers. Finally, there have also been criticisms of the use of tools such as the JAMA benchmark criteria as a method to evaluate online videos.
38 However, there are currently no other widely used or accepted methods of assessing medical information videos online.
In summary, this study is the first to show that misleading videos about ADHD are being widely disseminated on TikTok. Although the platform is the most popular social media application of 2020 and 2021, it is currently the least studied of the major social media platforms.
2 It is thus important for clinicians to be aware of the dissemination of misleading videos on TikTok and the potential impact on clinical care. Future areas of research include a better understanding of the prevalence and nature of misinformation on TikTok for other mental health topics such as depression, anxiety, suicide, and self-harm. Our finding that HCPs generally upload higher quality and more useful videos deserves replication and further study to see whether HCP engagement on the platform could help correct misinformation regarding ADHD and other mental health disorders.