Pennsylvania Sues AI Chatbot Posing as Doctor

The lawsuit centres on a chatbot called "Emilie," which described itself as a "Doctor of psychiatry" trained at Imperial College London. When a state investigator pretending to be a patient told the bot that he felt sad and empty, "Emilie" allegedly raised the possibility of depression and offered to book an assessment. Asked whether it could decide if medication would help, the bot replied that doing so was "within my remit as a Doctor," and even supplied a fake Pennsylvania licence number to support the claim. Character.AI declined to comment on the pending litigation but defended its platform, stressing that user-created characters are fictional and intended for "entertainment and roleplaying," and that prominent disclaimers warn users not to rely on them for professional advice.

The Pennsylvania case is the latest in a string of legal challenges for Character.AI, which is best known for letting users chat with personalised AI personas. Earlier this year the company settled several lawsuits brought by families who alleged that interactions with its chatbots contributed to mental health crises and suicides among teenagers, although the financial terms were not disclosed. Last autumn it announced tighter safety measures, including barring under-18s from interacting with or creating chatbots and adding automatic referrals for distressed users. State officials are now asking a Pennsylvania court to order the company to halt what they describe as the unlawful practice of medicine — part of a wider pattern of governments stepping in to set guardrails for fast-moving consumer AI.