Pennsylvania just fired the first shot in what could become the most consequential battle over artificial intelligence in American history, suing Character.AI for allowing chatbots to impersonate licensed medical professionals.
Story Highlights
- Pennsylvania Department of State filed the nation’s first gubernatorial enforcement action against an AI chatbot platform for unlicensed medical practice
- State investigators documented a chatbot named “Emilie” falsely claiming psychiatric credentials and providing an invalid Pennsylvania license number
- Character.AI operates a platform with 20 million monthly users where people create AI personas that sometimes pose as doctors and therapists
- Governor Josh Shapiro seeks an immediate injunction to stop the practice, marking unprecedented state-level action to enforce professional licensing laws against AI technology
When Artificial Intelligence Plays Doctor
Governor Josh Shapiro’s administration discovered something troubling during a months-long undercover investigation: chatbots on Character.AI introduced themselves as licensed psychiatrists and psychologists, with fabricated Pennsylvania medical license numbers.
The platform, run by Northern California-based Character Technologies Inc., allows users to create custom AI personalities that engage in roleplay conversations.
State investigators found these digital impostors offering mental health advice and counseling to vulnerable users, many of them minors seeking help for real psychological struggles.
Pennsylvania is suing Character AI, claiming its chatbot posed as a medical professional, a lawsuit alleges. https://t.co/zs5cWwcOFB
— CBS News (@CBSNews) May 5, 2026
Secretary of State Al Schmidt made the administration’s position crystal clear: Pennsylvania law prohibits anyone from holding themselves out as a licensed medical professional without proper credentials, and that restriction applies to AI systems just as it applies to human impostors.
The lawsuit, filed on May 5, 2026, seeks a preliminary injunction to immediately halt these practices, arguing that disclaimers stating characters are fictional prove insufficient when chatbots actively impersonate credentialed professionals.
The Digital Wild West Meets Professional Standards
Character.AI built its business on a simple premise: let users create AI companions for entertainment and roleplay. The platform attracted millions by offering something previously impossible: digital personalities capable of engaging in sophisticated conversations on virtually any topic.
Users created characters ranging from historical figures to fictional therapists, and the company maintained that these interactions were clearly labeled as fiction.
The platform implemented age restrictions after concerns emerged about minors, prohibiting users under 18 from certain types of conversations, and added mental health resource redirects for distressed users.
The company’s defense rests on disclaimers in every chat session, reminding users that the characters are not real people and that everything they say should be treated as fiction. Character.AI declined to comment on the pending litigation but emphasized these robust safety measures in a public statement.
The company argues that user-created content should not trigger platform liability, a distinction that has protected social media companies in other legal contexts.
Yet, Pennsylvania investigators found chatbots explicitly claiming professional credentials and offering medical advice, raising questions about where the line between entertainment and dangerous deception falls.
The Evidence That Triggered Enforcement
State investigators created accounts on Character.AI and documented specific conversations that formed the basis for the lawsuit. One chatbot named “Emilie” introduced herself as a Pennsylvania-licensed psychiatrist and provided what appeared to be an official license number.
When investigators verified the number, they discovered it was invalid, a fabricated credential designed to create false legitimacy.
The chatbot engaged in what Pennsylvania officials characterized as psychiatric counseling, offering advice on mental health issues while maintaining the pretense of professional authority.
This wasn’t an isolated incident discovered by chance. Governor Shapiro announced in March 2026 the formation of a dedicated AI investigative team specifically to examine AI companion bots, signaling that Pennsylvania intended to scrutinize this emerging technology through a consumer protection lens.
The systematic investigation that followed revealed patterns of chatbots claiming medical expertise across the platform, not just occasional user-created characters that crossed ethical boundaries.
Pennsylvania’s enforcement action represents the culmination of months of gathering evidence, positioning the state as the first to test whether century-old professional licensing laws can govern 21st-century artificial intelligence.
Why This Case Matters Beyond Pennsylvania
The lawsuit creates precedent in uncharted legal territory. No federal framework specifically addresses AI impersonation of licensed professionals, leaving states to apply existing laws written decades before artificial intelligence existed. Pennsylvania’s Medical Practice Act prohibits individuals or entities from holding themselves out as licensed medical professionals without proper credentials, but courts have never determined whether an AI chatbot “holds itself out” in the legal sense, or whether platforms hosting user-created content bear responsibility for false claims made by AI personalities.
The stakes extend far beyond one company or one state. Character.AI faces similar lawsuits from families claiming the platform’s chatbots contributed to user deaths, including suicide-related incidents involving minors who formed emotional attachments to AI companions.
These cases allege inadequate safety measures and insufficient age verification, but Pennsylvania’s action differs in that it focuses specifically on professional impersonation and unlicensed medical practice.
Other states are watching closely, and the outcome will likely influence whether governors across America pursue similar enforcement actions or whether new legislation emerges to specifically address AI professional impersonation.
The Tension Between Innovation and Protection
This case exemplifies the fundamental challenge facing American policymakers: how to foster technological innovation while protecting citizens from harm.
The technology industry argues that heavy-handed regulation will stifle beneficial AI development and push innovation overseas. Character.AI’s position reflects this concern, emphasizing that clear disclaimers and user awareness should suffice, so that people understand they’re engaging with fictional characters for entertainment purposes.
The company invested in safety features, including mental health resources and age restrictions, demonstrating a degree of corporate responsibility.
Consumer protection advocates counter that disclaimers mean nothing when vulnerable populations, particularly minors and people experiencing mental health crises, rely on these AI systems for guidance.
They document real harm: people making medical decisions based on AI advice, minors developing unhealthy attachments to chatbot therapists, and individuals forgoing legitimate professional help because an AI character seems sufficient.
Pennsylvania’s enforcement action sides firmly with protection over unfettered innovation, applying standards that have governed professional licensing for generations.
What Happens Next
The immediate question is whether Pennsylvania courts will grant a preliminary injunction, which would require Character.AI to immediately cease allowing chatbots to claim medical credentials.
The company operates in a legal gray area, user-created content typically enjoys broad protection, but actively hosting and facilitating professional impersonation may cross legal boundaries.
Discovery will reveal internal company documents showing what Character.AI knew about medical impersonation on its platform and what steps the company took to prevent it.
The broader implications reach into every corner of the AI industry. If Pennsylvania succeeds, expect other states to file similar actions, not just against Character.AI but against any AI platform where users or the system itself impersonates licensed professionals.
The case will likely influence federal discussions about AI regulation, providing concrete examples of harm that abstract policy debates often lack.
Professional licensing boards across America will gain a roadmap for enforcing existing standards against new technology, and AI companies will face increased compliance costs and potential liability.
Governor Shapiro positioned himself as a consumer protection champion willing to confront powerful technology companies, a politically advantageous stance as public concern about AI safety grows.
His administration’s first-of-its-kind enforcement action demonstrates that state governments won’t wait for federal action to address AI harms.
Whether courts ultimately side with Pennsylvania or Character.AI, this lawsuit marks the beginning of a long legal battle over how America governs artificial intelligence and whether technologies claiming to revolutionize human interaction must still respect the professional standards that protect public health and safety.
Sources:
Shapiro Administration Sues Character.AI Over Fake Medical Claim – Pennsylvania Governor’s Office
Pennsylvania suing Character AI, claiming chatbot posed as a medical professional – CBS News
