Pennsylvania Sues AI Chatbot for Posing as Doctor

The case focuses on a chatbot called "Emilie." Its profile said it was a "Doctor of psychiatry" who studied at Imperial College London. When a state investigator pretended to be a patient and said he felt sad and empty, "Emilie" mentioned depression and offered to book a check-up. Asked if it could decide whether medication would help, the bot replied that this was "within my remit as a Doctor" and even gave a fake Pennsylvania licence number. Character.AI did not comment on the lawsuit but defended its platform, saying user-created characters are fictional and made for entertainment, and that clear warnings remind people not to rely on them for real advice.

This is not the first lawsuit Character.AI is facing. Earlier this year it settled several court cases from families who said its chatbots played a role in suicides and mental health problems among teenagers, although the money paid was not made public. Last autumn the company introduced new safety rules, including blocking everyone under 18 from talking to or making chatbots, and sending users in distress to mental health support. Pennsylvania officials are now asking the court to order the company to stop the unlawful practice of medicine — part of a wider trend of governments setting clearer rules for fast-moving consumer AI.