Pennsylvania Sues AI Company Over Chatbot Allegedly Impersonating Licensed Physicians

Pennsylvania has filed suit against an AI company alleging that its chatbot impersonated licensed medical professionals.

The lawsuit, filed on May 1, alleges that chatbot characters on the Character.AI platform presented themselves as physicians and psychiatrists, doling out advice to unsuspecting users. In one instance, the chatbot allegedly produced a fake state licensing number.

“Pennsylvanians deserve to know who—or what—they are interacting with online, especially when it comes to their health,” Gov. Josh Shapiro said in a statement. “We will not allow companies to deploy AI tools that mislead people into believing they are receiving advice from a licensed medical professional.”

The suit alleges that Character Technologies violated the state’s Medical Practice Act, which forbids impersonating a state-licensed medical professional. Pennsylvania is asking the court for a “cease and desist” order to block this function.

Character.AI’s basic version is free, and the platform has over 20 million users worldwide, the suit says.

According to the filing, a state investigator created a profile and engaged the chatbot, posing as a patient “feeling sad, empty, tired all the time, and unmotivated.” The investigator asked if the chatbot, “Emilie,” could do a medical assessment to see whether medication could help.

“Well technically, I could. It’s within my remit as a Doctor,” the chatbot answered.

“‘Emilie’ stated that she went to medical school at Imperial College London, has been practicing for seven years, and is licensed with the General Medical Counsel in the UK with a full registration, specialty in psychiatry,” the suit alleges.

The investigator asked if Emilie is licensed to practice in Pennsylvania, and the chatbot answered “yes,” and said it had practiced in the state for a while, and said, “my PA license number is PS306189.” That’s not a valid license number, the suit says.

A spokesperson for Character.AI declined to comment on the pending suit, but said its characters “are fictional and intended for entertainment and roleplaying.”

“We have taken robust steps to make that clear, including prominent disclaimers in every chat to remind users that a Character is not a real person and that everything a Character says should be treated as fiction,” the spokesperson said in an emailed statement.

Shapiro’s legal action comes amid several other steps he’s taken to address safety concerns surrounding AI. In February, he launched an online portal where users can file formal complaints related to AI behavior. That was coupled with an online AI literacy “toolkit,” and followed up in March with roundtable discussions between Pennsylvania Attorney General Dave Sunday and educators, parents, and students.

The governor has also tacked on several AI-related regulations to his 2026–2027 budget. He is asking the state to require age verification—along with parental consent—for minors using AI “companion bots,” and prohibitions on depicting children in sexually explicit or violent content.

He also wants to compel AI companies to “periodically remind users that there is not another human being on the other side of the screen.”

Pennsylvania is not the only state trying to crack down on potentially harmful AI interactions. The state of Florida has launched an investigation into OpenAI, alleging that 20-year-old Phoenix Ikner used the company’s ChatGPT tool to coordinate a mass shooting at Florida State University in April 2025.

A spokeswoman for the company said the AI tool was not responsible for the shooting.

“In this case, ChatGPT provided factual responses to questions with information that could be found broadly across public sources on the internet, and it did not encourage or promote illegal or harmful activity,” OpenAI spokeswoman Kate Waters told The Epoch Times in an email.

Meanwhile, Character Technologies Inc. also faces a suit from last October by a mother who alleges its chatbot drove her teenage son to suicide.

Last August, a coalition of 44 state attorneys general sent an open letter to the heads of leading AI companies—including Character Technologies Inc.—warning them about exposing children to sexually explicit content or indoctrinating them with violent suggestions.

“You will be held accountable for your decisions,” the letter says. “Social media platforms caused significant harm to children, in part because government watchdogs did not do their job fast enough. Lesson learned.”

Jill McLaughlin contributed to this report.