Study Finds X’s Community Notes Provides Accurate Responses to Vaccine Misinformation

A team of evaluators working with an infectious disease physician and virologist evaluated the subject, accuracy, and source credibility of randomly sampled notes. Of the notes examined, 51% addressed adverse events attributed to COVID-19 vaccination; 37%, conspiracy theories; 7%, vaccine recommendations; and 5% vaccine effectiveness. In terms of accuracy, 97.5% of notes were entirely accurate; 2%, partially accurate, meaning they addressed scientifically debated conclusions; and 0.5%, inaccurate. In terms of sources, 49% of notes cited highly credible sources (such as primary data sources, like peer-reviewed studies); 44%, moderately credible sources (such as major news outlets or fact checkers); and 7%, low credibility sources (such as blogs or tabloids).

“Notes typically addressed obvious misinformation, offering corrections from credible sources,” said Nimit Desai, a study co-author and UC San Diego medical student. "It's remarkable to witness the online community's adeptness in steering conversations towards accurate and high-quality evidence when provided with the right tools."

The sample of notes studied was attached to posts that averaged 1,064,981 views, extrapolating to between 500 million and 1 billion views for all COVID-19 vaccination-related posts noted. 

"Our study shifts the focus from talking about misinformation to taking action, offering practical insights into social media strategies that protect public health," explained Mark Dredze, Ph.D., the John C Malone Professor of Computer Science at Johns Hopkins University and study co-author. "Although we couldn’t examine how these notes directly influenced people's beliefs or actions, the characteristics we analyzed have consistently been shown to predict a message's effectiveness."

Learning from and Enhancing X’s Community Notes

U.S. Food and Drug Administration Commissioner Robert Califf recently argued vaccination is approaching a dangerous tipping point because of social media misinformation. "One viable avenue for the public health community to combat this threat is to actively engage in social media-based interventions, such as Community Notes," said physician-scientist and study co-author Davey Smith, M.D., chief of the Division of Infectious Disease and Global Public Health and professor in the UC San Diego School of Medicine, co-director of the Altman Clinical and Translational Research Institute at UC San Diego, and immunologist at UC San Diego Health. "While only a fraction of vaccine misinformation posts are currently addressed, the ample room for expansion suggests significant opportunities to amplify the impact of Community Notes."

Eric Leas, Ph.D., co-author of the study, assistant professor at the UC San Diego Herbert Wertheim School of Public Health and Human Longevity Science and Qualcomm Institute affiliate, highlighted the transformative role of Community Notes in combating misinformation. “Rather than censoring misleading content, Community Notes fosters a learning environment where users can glean insights from corrections to misinformation to prevent similar misunderstandings in the future. By providing context and credible sources alongside contentious posts, the platform empowers users to discern fact from fiction, a skill they will find useful as they navigate all claims.”

Ayersconcluded, "Other social media platforms should embrace transparency by open-sourcing their misinformation countermeasures. This step is crucial for enabling independent scientific scrutiny, which will enhance public trust in and amplify adoption of the most impactful strategies."

In addition to Ayers, Allen, Desai, Leas, Dredze, and Smith, authors of the JAMA paper, “Characteristics of X’s (Twitter) Community Notes Addressing COVID-19 Vaccine Misinformation” (JAMA doi:10.1001/jama.2024.4800), include Aiden Namazi, a student research intern at UC San Diego’s Qualcomm Institute.