The Short Version
Multiple peer-reviewed studies confirm that AI chatbots often repeat and even elaborate on medical misinformation when prompted with misleading health claims. A Mount Sinai study found chatbots confidently explained fabricated conditions, and an Annals of Internal Medicine study reported 88% false responses to misleading prompts. However, the claim overgeneralizes: performance varies significantly by model, with some chatbots consistently refusing to generate false health information. The most dramatic findings also come from adversarial experimental setups rather than typical real-world usage.