Friday, Jul 24, 2026 The claims desk. Receipts included. POWERED BY LENZ
IsThis

HEALTH

The Claim

AI chatbots frequently repeat medical misinformation when prompted with misleading health claims.

The Short Version

Multiple peer-reviewed studies confirm that AI chatbots often repeat and even elaborate on medical misinformation when prompted with misleading health claims. A Mount Sinai study found chatbots confidently explained fabricated conditions, and an Annals of Internal Medicine study reported 88% false responses to misleading prompts. However, the claim overgeneralizes: performance varies significantly by model, with some chatbots consistently refusing to generate false health information. The most dramatic findings also come from adversarial experimental setups rather than typical real-world usage.

Caveats

  • The word 'frequently' does not apply uniformly — some AI models consistently avoid generating health misinformation, while others are highly susceptible, making this a model-dependent pattern rather than a universal one.
  • The most cited statistics (e.g., 88% false responses) come from controlled adversarial experiments designed to stress-test chatbots, not from naturalistic user interactions, so real-world frequency may differ.
  • AI model safeguards are evolving rapidly; findings from studies conducted in 2023–2025 may not reflect the current capabilities or guardrails of the latest chatbot versions.

The Receipts

  1. AI Chatbots Can Run With Medical Misinformation, Study Finds

    Mount Sinai Health System

  2. AI chatbots and (mis)information in public health - PMC

    PMC (PubMed Central)

  3. Evaluation of the readability, quality, and accuracy of AI chatbot responses to health information queries

    PubMed Central (NIH)

  4. Medical Misinformation in AI-Assisted Self-Diagnosis: Development ...

    JMIR Formative Research

  5. AI Chatbots as Health Information Sources — The Monitor

    KFF (Kaiser Family Foundation)

  6. Use of generative AI chatbots and wellness applications for mental health

    American Psychological Association (APA)

  7. AI chatbots could spread 'fake news' with serious health consequences - University of South Australia

    University of South Australia

  8. Leveraging Chatbots to Combat Health Misinformation for Older Adults

    JMIR Formative Research

  9. Health Misinformation: AI Generated Content - Subject Guides

    Cleveland Clinic LibGuides

  10. Video | The Misuse of AI Chatbots in Healthcare: Risks, Realities, and

    ECRI

Filed Under

AI chatbots

More Fact Checks