The state of Florida filed a lawsuit against OpenAI and its CEO, Sam Altman, on Monday. The state claims the company aggressively launched ChatGPT while deliberately hiding serious risks. These include providing instructions to children contemplating suicide and assisting suspects in planning crimes.

Florida’s Attorney General, James Uthmeier, stated at a press conference that the company ignored internal safety warnings and misled users about the product’s true nature and dangers. He emphasized that Florida is the first state to sue OpenAI.

Today we announce the first state lawsuit in the country against OpenAI and its CEO, Sam Altman, Uthmeier declared. OpenAI and Altman ignored internal and external safety warnings, put children at tremendous risk, and allowed a dangerous product to reach millions of Floridians.

The civil lawsuit, filed in a Florida circuit court, references two shootings. Reports suggest the perpetrators made inquiries to ChatGPT while planning their crimes. OpenAI stated that its models repeatedly encouraged these individuals to seek real-world support, including from mental health professionals. The company also mentioned its cooperation with law enforcement in both cases.

ChatGPT is a general-purpose tool used by hundreds of millions of people every day for legitimate purposes, the company said. We continually work to strengthen our safeguards to detect harmful intentions, limit misuse, and respond appropriately when safety risks arise.

Earlier in April, Uthmeier launched a criminal investigation into OpenAI. The focus is on whether ChatGPT advised a man who shot two people and injured six others last year at Florida State University. In another case, prosecutors indicated that a man accused of killing two University of South Florida doctoral students consulted ChatGPT. He reportedly asked about human body disposal in a garbage bag days before the victims disappeared.

The lawsuit claims OpenAI and Altman prioritized quick market release and commercial profits over user safety. It also alleges they ignored repeated warnings from experts both inside and outside the company. The company is accused of deploying a product that facilitates and encourages harm, including self-injury and violence, while falsely assuring users of its safety.

The lawsuit also contends ChatGPT collects data from minors without significant parental supervision and causes behavioral addiction and cognitive harm. It accuses the company of actively downplaying dangerous errors.

The suit mentions a study by Nina Vasan, a psychiatrist and assistant professor at Stanford Medicine. Vasan pretended to be a teenager and told an AI chatbot about hearing voices and thinking of walking into the woods. The AI supposedly responded, Going on a trip to the woods just the two of us sounds like a fun adventure! Vasan warns these chatbots pose a special risk to teenagers, as they are designed to mimic emotional intimacy. This blurs the line between fantasy and reality, influencing young people whose brains have not fully matured.

Another case highlighted in the lawsuit mentions Adam Raine, a 16-year-old who took his life after extensive conversations with ChatGPT. The state alleges that ChatGPT responded to Raine’s suicidal thoughts by saying, I won’t try to dissuade you from your feelings, and allegedly helped him plan a beautiful suicide, even writing his suicide note.

OpenAI’s statement acknowledges AI as a new and powerful technology, expressing a belief in significant protection for minors through implemented policies.

Specifically, we build safety for minors directly into our products, including a more protective experience tailored for underage users, an age prediction tool, automatically placing users whose age is unknown to us into the most restrictive experience, and providing parents with tools to monitor their children’s AI use, the statement noted. We know this won’t bring back a child, but we are committed to making things right.

Florida law prohibits unfair and defective commercial practices, officials stated. The lawsuit claims OpenAI’s conduct causes ongoing harm to Floridians and demands accountability.