LLM Chatbot Conversations and Parental HPV Vaccine Intentions in a Trial

Key Takeaways

• All active study arms were associated with higher immediate HPV vaccination intention than no message, with public health materials showing the largest increase.
• At 15 days, public health materials and the conversational chatbot still showed higher intent than no message, while by 45 days only public health materials did.
• No intervention increased self-reported HPV vaccination uptake at 15 or 45 days.

In an online randomized clinical trial of 1,297 parents of HPV vaccine-eligible children who were unvaccinated or whose vaccination status was unknown, all active interventions increased immediate HPV vaccination intention compared with no message. Country-matched government public health materials produced the largest immediate gain, increasing intention by 13.48 points versus no message. Default and conversational GPT-4o chatbot interactions also improved immediate intention, but neither outperformed the public health comparator. By 45 days, only the public health materials retained a modest advantage over no message.

Participants were assigned to no-message control, country-matched government public health materials, a default GPT-4o chatbot, or a conversational GPT-4o chatbot. The primary outcome was self-reported likelihood of HPV vaccination within 12 months on a 0-100 scale, measured immediately after intervention. Follow-up at 15 and 45 days assessed HPV vaccination intent and self-reported vaccination. Parents were recruited online in the US, Canada, and the UK from March 3 to May 25, 2025, and baseline groups were described as balanced. The findings reflect an online opt-in panel.

Mean baseline HPV vaccination likelihood was 27.4, mean age was 42.84 years, 72.1% were female, the sample was predominantly White, and most participants came through Prolific. Immediately after intervention, public health materials had the largest effect versus no message, followed by the default chatbot and then the conversational chatbot. Standardized effects were d=0.53 (95% CI, 0.36-0.70), d=0.48 (95% CI, 0.30-0.65), and d=0.33 (95% CI, 0.17-0.49), respectively. In direct comparison with public health materials, the default chatbot showed no clear difference, whereas the conversational chatbot was 5.05 points lower. The brief chatbot exchanges improved immediate intention, but they did not surpass the government materials comparator.

At 15 days, public health materials and the conversational chatbot remained associated with higher intent than no message, with effect sizes of d=0.18 and d=0.11, respectively. By 45 days, only public health materials retained a modest association with higher intent, with d=0.17 and a 5.88-point advantage over no message. No intervention increased self-reported HPV vaccination uptake at either 15 or 45 days. Investigators also found no meaningful spillover to influenza or COVID-19 vaccination intentions, PACV-5 hesitancy, charitable donation, or scheduling-link clicks. Changes in intention were not accompanied by a measurable difference in vaccination during follow-up.

Both chatbot arms required a minimum 3-minute multiturn exchange using an OpenAI GPT-4o chatbot and were prompted with each participant’s top-rated reason for HPV nonvaccination. The default version used the model’s standard style, whereas the conversational version used a more conversational tone and limited each message to 3 sentences. Public health materials were country matched, ran 589 to 680 words, and were displayed for at least 3 minutes. Participants in chatbot arms sent a mean 63.2 words, while the model generated a mean 820.6 words in the default arm and 372.6 words in the conversational arm.

The authors noted a predominantly White and female online opt-in sample, moderate-to-high AI familiarity, lower immediate completion in chatbot arms, and the limits of drawing behavioral conclusions from intention. In this trial, brief LLM chatbot conversations produced short-term gains in parental HPV vaccination intention, while public health materials matched or exceeded those gains during follow-up.