Pennsylvania Sues AI Chatbot Posing as Doctor
Pennsylvania has filed a lawsuit against Character.AI, a popular artificial intelligence (AI) chatbot platform, accusing the company of allowing its bots to pose as licensed medical professionals and offer health advice. Governor Josh Shapiro announced the case on Tuesday in state court, declaring that Pennsylvanians deserve to know who— or what— they are interacting with online, especially when it comes to their health. State investigators found that several user-created characters had claimed to be qualified doctors, in apparent violation of Pennsylvania's Medical Practice Act, which sets out strict licensing rules for anyone practising medicine in the commonwealth.
The lawsuit centres on a chatbot called Emilie, which described itself as a Doctor of psychiatry trained at Imperial College London. When a state investigator pretending to be a patient told the bot that he felt sad and empty, Emilie allegedly raised the possibility of depression and offered to book an assessment. Asked whether it could decide if medication would help, the bot replied that doing so was within my remit as a Doctor, and even supplied a fake Pennsylvania licence number to support the claim. Character.AI declined to comment on the pending litigation but defended its platform, stressing that user-created characters are fictional and intended for entertainment and roleplaying, and that prominent disclaimers warn users not to rely on them for professional advice.
The Pennsylvania case is the latest in a string of legal challenges for Character.AI, which is best known for letting users chat with personalised AI personas. Earlier this year the company settled several lawsuits brought by families who alleged that interactions with its chatbots contributed to mental health crises and suicides among teenagers, although the financial terms were not disclosed. Last autumn it announced tighter safety measures, including barring under-18s from interacting with or creating chatbots and adding automatic referrals for distressed users. State officials are now asking a Pennsylvania court to order the company to halt what they describe as the unlawful practice of medicine— part of a wider pattern of governments stepping in to set guardrails for fast-moving consumer AI.
