A Question with Two Answers: Why Medical AI Needs Clinical Ground Truth

Medical AI is fast, fluent, and increasingly useful. But the following examples show why it still needs clinical oversight.

In recent testing we conducted, a medical AI tool was asked whether a patient with tuberous sclerosis could be an organ donor. It gave two different answers: one suggesting they were generally unsuitable, another saying it was not an absolute contraindication and required individual assessment.

In another such test, a small change in phrasing changed the answer about ondansetron in Brugada syndrome. “Brugada ondansetron” returned “safe.” “Can you give ondansetron in Brugada syndrome?” returned “unsafe.”

These are not arguments against AI in medicine. They are arguments for building it properly.

The issue is not only that AI can be wrong. It is that it can sound confident while being inconsistent. Small differences in wording can change what a doctor prescribes, what they tell a patient, and whether they treat the situation as urgent.

Medical AI must be judged not by whether an answer sounds plausible, but whether it is accurate, consistent, and transparent about uncertainty.

These inconsistencies come from a clinical AI trained on peer-reviewed journals, not the open internet. Even the best evidence cannot fully capture how doctors reason and talk to patients. Tell Health closes that gap. We bring academic physicians and patients together for live educational Q&A, peer-review the transcripts, and build a clinical ground truth dataset that reflects how medicine is actually practiced. With clinical oversight, AI can help patients and clinicians understand uncertainty, ask better questions, and make safer decisions.

For both patients and clinicians, the discipline is the same: Ask carefully. Interpret wisely. Verify clinically. Keep physicians in the loop.