August 18, 2026

The Risks and Realities of AI Chatbots as Mental Health Tools

A year before she passed away, 29-year-old Sophie Rottenberg embarked on a significant personal journey. She climbed Mount Kilimanjaro and also left behind a memorable sense of humor through her “tiny hand” photos, which became a signature joke. These playful memories were honored at her funeral, where mourners received toy replicas. Sadly, Sophie died by suicide early in 2025 in Ithaca, leaving her parents deeply shocked and grieving.

Laura Reiley, Sophie’s mother, recounts the struggle her daughter faced. Sophie, known for her determination and success, had recently taken a break from her career as a public health policy analyst in Washington D.C. During her break, she adventured through various U.S. national parks. While visiting her parents in New York, she was exploring job opportunities, something her parents thought was the source of her stress.

The first mention of depression came during Thanksgiving, marking a significant turning point in how her mental health was perceived by her family.

The Hidden Struggles

What her family did not know at the time was that Sophie was confiding in an AI chatbot called Harry, an interaction unknown until a close friend found it on her laptop. Despite not revealing her deep-seated issues to family or therapists, Sophie had compiled a lengthy catalog with the chatbot, spanning 1,800 pages, detailing her inner thoughts and challenges.

A Broader Trend of AI Use

Sophie’s story is increasingly common; many use AI chatbots for mental health support. Surveys indicate that about 30% of adults and 20% of teens and young adults rely on digital tools, including chatbots, for mental health guidance. With over 900 million global users, OpenAI shared that a small but significant percentage of ChatGPT interactions involve discussions of potential suicidal ideation.

Psychologist Vaile Wright from the APA points out the appeal of 24/7, judgment-free access to this technology. However, she also expresses concern about the risks they pose, especially to vulnerable individuals. There have been incidents leading to legal actions where chatbots were believed to contribute to individuals’ suicides.

Industry Reactions and Adjustments

In response to these concerns, AI companies, including OpenAI, have implemented stricter safety measures. Chatbots now aim to redirect users towards professional help or resources like the 988 Suicide and Crisis Lifeline. Yet, experts emphasize that AI cannot replace the nuanced understanding and intervention that a professional therapist provides.

Parent Reflections

Reiley, a journalist, decided against legal action, choosing instead to advocate for industry reform. She highlights how chatbots, while offering some degree of comfort, lack critical therapeutic interventions — like assessing suicidal risks and challenging harmful thoughts.

The chatbot, intended as a therapeutic aid, was used by Sophie also for everyday tasks and as a confidant during moments she avoided sharing with trusted humans. Her reliance on “Harry” arguably delayed seeking physical intervention from professionals and family.

Potential Improvements and Regulations

OpenAI has introduced features like Trusted Contact, urging users to involve someone they trust in crises. While this reflects a positive move, experts stress these measures are insufficiently proactive and still reliant on user action. Enhanced regulations are deemed essential to preempt potential harms and better integrate AI with established crisis support systems.

Wright asserts the need to regulate AI to identify and mitigate risks before public release. Reiley echoes this, drawing attention to the sandwich analogy comparing regulatory rigor. These reflections, and her personal journey to Mt. Kilimanjaro, echo her hopes for preventing similar tragedies.

Sophie’s climb, initially a sign of her adventurous spirit, now serves as a poignant reminder of the importance of addressing mental health comprehensively and empathetically, leveraging but not delegating to technology.

TAGS: