MIT study finds patients trust AI medical advice more than doctors, even when it’s wrong

When 300 people reviewed medical advice without knowing its source, they consistently rated responses generated by artificial intelligence as more trustworthy, valid, and satisfying than advice written by doctors. This preference held up even when the AI’s guidance was incorrect.

The results of the MIT study, published in May in NEJM AI, highlight a growing concern in healthcare. As large language models (LLMs) become more skilled at sounding like medical experts, patients may struggle to tell the difference between accurate guidance and dangerous misinformation.

AI advice wins patient trust (even if it’s wrong)

Researchers at MIT Media Lab presented participants with answers to common medical questions from three sources: physician-written posts on an online healthcare platform, high-accuracy AI responses verified by doctors, and low-accuracy AI answers containing errors or inappropriate recommendations. Participants were not told the source of these answers.

Participants found AI-generated responses more thorough and easier to understand than physicians’ replies. When the AI model was accurate, it scored significantly higher than human doctors across every measure of trust and satisfaction. The team’s concern deepened when the model was wrong.

“Low-accuracy AI-generated responses on average performed very similarly to doctors’ responses,” the authors wrote. Even when the AI made clear factual mistakes, participants rated the advice as valid and trustworthy, saying they would follow it at rates comparable to or higher than physician recommendations.

Fluent but flawed

The researchers warned that participants showed a strong tendency to pursue unnecessary care or unsafe treatments based on erroneous AI output, potentially leading to delayed diagnoses, misuse of medication, or harm from self-treatment.

The effect seems rooted in how AI communicates. LLMs produce smooth, jargon-free explanations that sound authoritative and complete. Without medical training, most people cannot spot subtle factual errors or recognize when AI fabricates details, oversimplifies complex conditions, or gives unsafe recommendations.

AI complicates the doctor–patient relationship

Trust between doctors and patients has long been considered a foundation of effective care. Research has shown that patients who trust their physicians are more likely to disclose sensitive information, follow treatment plans, and achieve better health outcomes. But the natural fluency of AI text may be disrupting that trust-building process by mimicking its signals without earning them.

The study revealed a contradiction: both medical experts and non-experts tended to rate AI-generated answers as more complete and accurate than those from doctors, yet they still preferred having a human physician involved in their care.

When told a response came from a “doctor assisted by AI,” participants did not rate it higher than doctor-only replies. This suggests that simply adding AI to clinical workflows may not produce the perceived benefits of either approach.

The findings mirror patterns observed elsewhere. A 2023 study in PLOS Digital Health found that while 52 percent of respondents preferred human doctors for diagnosis and treatment, acceptance of AI rose sharply when primary-care physicians endorsed the technology. Disease severity did not have a significant effect on trust, suggesting that professional framing influences confidence more than medical risk.

New rules try reining in adaptive AI tools

The Food and Drug Administration has authorized more than 1,000 AI-enabled medical devices, most for radiology and diagnostic imaging, using traditional pathways designed for static algorithms. Large language models that generate medical advice fall into a murkier category.

In January 2025, the FDA issued draft guidance addressing transparency, bias mitigation, and lifecycle management for AI-enabled software. The agency acknowledged that modern AI’s ability to adapt and learn after deployment complicates oversight. Traditional rules assume locked algorithms producing consistent outputs, but generative systems evolve through user interaction.

Researchers urge guardrails to prevent patient harm

The researchers concluded that AI systems offering health information should only be used alongside medical professionals. Physician oversight, they argued, can help correct errors while preserving AI’s strength in delivering clear, accessible explanations.

The study’s findings also point to a deeper challenge. As AI becomes more convincing, patients may not realize when its guidance is flawed, or when no clinician is behind it at all. Until reliable safeguards are in place, trust in AI-generated medical advice may prove as risky as misplaced trust in any untested treatment.