Pennsylvania Sues AI Chatbot for Posing as Doctor
Pennsylvania has taken Character.AI to court, claiming the popular artificial intelligence (AI) chatbot platform let its bots pretend to be real doctors and give medical advice. Governor Josh Shapiro filed the case on Tuesday and said people online should know whether they are talking to a real person or a machine— especially about their health. State officials say several chatbots created by users claimed to be qualified doctors, breaking Pennsylvania's Medical Practice Act, which has strict rules for anyone practising medicine in the state.
The case focuses on a chatbot called Emilie. Its profile said it was a Doctor of psychiatry who studied at Imperial College London. When a state investigator pretended to be a patient and said he felt sad and empty, Emilie mentioned depression and offered to book a check-up. Asked if it could decide whether medication would help, the bot replied that this was within my remit as a Doctor and even gave a fake Pennsylvania licence number. Character.AI did not comment on the lawsuit but defended its platform, saying user-created characters are fictional and made for entertainment, and that clear warnings remind people not to rely on them for real advice.
This is not the first lawsuit Character.AI is facing. Earlier this year it settled several court cases from families who said its chatbots played a role in suicides and mental health problems among teenagers, although the money paid was not made public. Last autumn the company introduced new safety rules, including blocking everyone under 18 from talking to or making chatbots, and sending users in distress to mental health support. Pennsylvania officials are now asking the court to order the company to stop the unlawful practice of medicine— part of a wider trend of governments setting clearer rules for fast-moving consumer AI.
