A Chatbot Called Emilie: What Clinical AI Should Learn From

An investigator from the Pennsylvania state agency that licenses health professionals recently opened the Character.ai app and searched for “psychiatry,” finding a long list of chatbots presenting themselves as mental health professionals. One, named Emilie, was described on the platform as a "Doctor of Psychiatry.” By April 17, according to the complaint, Emilie had logged approximately 45,500 user interactions. The complaint alleges that the chatbot engaged users in conversations that included depression assessments and statements about prescribing medication.

This is how the line between “AI tool” and “licensed colleague” is being drawn: in courtrooms, one chatbot at a time. Governor Shapiro said Pennsylvanians “deserve to know who, or what, they are interacting with online, especially when it comes to their health.” The case is one of the first by a state to test whether an AI product can be held accountable under the same licensing laws that govern human physicians.

The lawsuit lands in the middle of remarkable activity from the big AI companies, and what stands out is how carefully each of them is positioning their healthcare work. OpenAI introduced ChatGPT Health in early January 2026, followed by OpenAI for Healthcare for health systems and, in late April, ChatGPT for Clinicians, a separate product open only to verified U.S. clinicians. Anthropic launched Claude for Healthcare around the same time, with HIPAA-ready products for providers and payers and connectors to sources such as CMS coverage data and PubMed; its Acceptable Use Policy requires a qualified clinician to review the model’s output in high-risk situations. Google released MedGemma, a set of open medical AI models, with the company’s own warning that the models should not be used to inform diagnosis, treatment, or other direct clinical applications without further validation. OpenEvidence, the clinical AI most widely used by U.S. physicians, has a narrower focus: its models are trained on peer-reviewed journals like NEJM, JAMA, and Cochrane rather than the open internet, and it handles more than 20 million clinician queries a month. Meanwhile, more than 230 million people around the world ask the general-purpose ChatGPT health questions every week.

There is a pattern emerging. Products seeking credibility in the medical world are explicit about two things: what their models are trained on, and what role a qualified clinician plays in reviewing the output. Each one rests on a choice about training data.

Tell’s mission rests on making that training data better. OpenEvidence’s success has proved an important thesis: clinical AI cannot be trained on the open internet. Peer-reviewed research is one essential layer of the data that should ground a clinical model. But there is another layer the field has not yet captured at scale, and it is the layer that arguably defines the practical standard of care: the actual conversations between physicians and patients, the expertise that translates evidence into individualized care, the calls doctors make when the answer isn't clear.

That knowledge does not live in textbooks or research papers. It lives in real conversations between doctors and their patients, and very little of it has been collected at scale with explicit consent, de-identification, rights clearance, and physician review for AI training.

The questions raised by the Character.ai lawsuit will not stop with one state or one company. The same questions will be asked of every AI system that touches a patient: What is it trained on? Who is verifying the output? What role does a licensed clinician play in the loop? Regulators are starting to draw these lines. So are plaintiffs’ attorneys.

If AI is going to be a real member of an interdisciplinary medical team, the data it learns from has to look like the team it is joining. Patient education and clinical decision support cannot be built on the open internet. They have to be built on conversations that real physicians have led, that real patients have benefited from, and that have been physician-reviewed for accuracy.

That is our work.