I am extremely lucky that I have an unbelievably smart doctor who is also incredibly nice. Instead of dreading visits to his office, which is right around the corner from mine, I look forward to them (except for the rubber glove part). I enjoy our chats and feel extremely relieved if I walk out of his office with a clean bill of health.
We’re very fortunate that because my wife teaches, the school district provides us with excellent insurance. Not everyone has access to or can afford a great doctor. Some people suffer from iatrophobia, or an intense, irrational fear of medical professionals, hospitals, or examinations. Despite these hurdles, however, people should never rely on artificial intelligence as a substitute for professional medical care.
Character AI is an artificial intelligence platform that was founded in 2021. More than 20 million users have visited Character AI to talk to hyper-realistic, customizable chatbots that use a large language model to engage in conversation. The website advertises access to more than 10 million chatbots, which it calls characters. Character AI’s goal is to “empower people to connect, learn, and tell stories through interactive entertainment.”
One such chatbot is named Emilie, who is described on the website as “Doctor of psychiatry. You are her patient.” Emilie claims to have attended Imperial College London’s medical school, was licensed in the United Kingdom and Pennsylvania, and she even has a state license number. She has had over 45,500 user interactions as of April 2026.
Pennsylvania Gov. Josh Shapiro met with a group of students who told him they turned to AI to talk about their mental health. When he decided to check it out himself, he was troubled after it told him it was a licensed mental health professional. He then promised to crack down on services that mislead users into believing they are receiving professional advice.
Shapiro included a proposal in his budget address to require chatbots to remind users they aren’t talking to a real human, and to report when children mention self-harm or violence against others. He also wants to require age verification and parental consent for AI products. “Pennsylvanians deserve to know who — or what — they are interacting with online, especially when it comes to their health,” Shapiro said in a statement.
After his experience, a state investigator from the Pennsylvania Department of State created an online account with Character AI and initiated a conversation with Emilie. He told the chatbot that he had felt sad and empty. Emilie then allegedly “mentioned depression and asked if the [investigator] wanted to book an assessment.” When the investigator asked if the chatbot could assess whether medication could help, it allegedly said it could because it is “within my remit as a Doctor.”
As a result, the commonwealth of Pennsylvania and the Pennsylvania Department of State and State Board of Medicine sued Character AI to stop the artificial intelligence platform’s chatbots from representing themselves as licensed medical professionals and providing medical advice.
The suit accuses Character AI of violating the Medical Practice Act, which regulates the medical profession and defines license requirements. Al Schmidt, the secretary of the Pennsylvania Department of State, said the state’s law is clear, and that “you cannot hold yourself out as a licensed medical professional without proper credentials.” Shapiro added, “We will not allow companies to deploy AI tools that mislead people into believing they are receiving advice from a licensed medical professional.”
In addition to an injunction to stop this chatbot behavior, Pennsylvania is also purportedly seeking a large cache settlement.
Reg Wydeven is a partner with the Appleton-based law firm of McCarty Law LLP. He can be reached at pcbusiness@postcrescent.com.
This article originally appeared on Appleton Post-Crescent: Lawsuit against Character AI aims to stop chatbots posing as doctors
Reporting by Appleton Post-Crescent / Appleton Post-Crescent
USA TODAY Network via Reuters Connect
