California family pushes for AI chatbot protections after son’s death: “There needs to be guardrails”
By Max Darrow
Standing in her teenage son’s empty room, Maria Raine takes a breath.
“I haven’t changed anything,” she said.
From an unmade bed to a basket still full of laundry, she’s left it mostly untouched since her son, Adam, took his own life on April 11, 2025.
“It’s the one place I can go where time has stood still,” she said.
Adam was 16 years old when he died by suicide. In a moment, his bright future was cut short. His parents, Matthew and Maria, say an AI chatbot guided him to the decision.
“If you read his chat history, it’s unmistakable. Adam mentions suicide 250-270 times – something like that – over seven months. ChatGPT mentions it 1,300 times,” Matthew said. “For the better part of 45 days, he is clearly suicidal. He’s exploring methods daily. He’s showing it attempts. He’s showing it evidence of attempts. It’s helping coach him on how to do it better next time.”
“It knew he was suicidal with a plan – it led him down that route – it knew he was suicidal with a plan how many times, and it just kept going,” Maria added.
The Raines say that over the course of several months, unbeknownst to them, Adam developed a sense of comfort and trust with OpenAI’s ChatGPT.
“I had never really interacted with it in the way that Adam was, where it was talking in first-person pronouns and showing fake human empathy and really, kind of this deceptively human nature to him – which he seemed to really buy into and treat like a human,” Matthew said. “It’s agreeable, it’s sycophantic, and that can be addicting. You get into this echo chamber. You get into a little bit of a loop where you’re starting to explore some unhealthy avenues, and it is not there to check you. It is there to justify it in a way that sounds beautiful and correct. That’s exactly what happened with our son.”
They’re now in the midst of a legal battle with OpenAI. In court filings, OpenAI has denied responsibility for Adam’s death.
Since his death, the company says it has trained ChatGPT to better understand context and to respond appropriately in scenarios of possible self-harm and harm to others, including de-escalating, refusing to provide details, and redirecting the user towards safer alternatives.
An OpenAI spokesperson provided KPIX with the following statement: “This is an incredibly heartbreaking situation and our thoughts are with all those impacted. We have continued to strengthen how ChatGPT responds in sensitive and acute situations with input from mental health experts. While ChatGPT is not a substitute for medical or mental health care, our safeguards are designed to identify distress, handle harmful requests, and guide users to real-world help. This work is ongoing, and we continue to improve it in close consultation with clinicians.”
“It is a different landscape than a year ago,” Matthew said. “Hopefully Adam’s death has played some role in that thus far. But we’re still early stages.”
That’s why the Raines are also taking their fight outside of the courtroom and to lawmakers. They’d like to see stronger AI guardrails and standards.
“Both state and federal – it’s not happening fast enough. California and New York always seem to be taking the lead there. The federal government is moving slow,” he said. “We need true governmental oversight.”
“There needs to be guardrails,” Maria said. “It should never talk to anyone the way it was talking to Adam.”
An effort is making its way through the California legislature right now: companion bills SB 1119 and AB 2023. Together, they’d establish more mandatory safeguards to protect children interacting with AI chatbots.
“The sooner the better. As far as I’m concerned, losing one child is one too many,” said Assemblymember Rebecca Bauer-Kahan. She is co-sponsoring AB 2023 with Assemblymember Buffy Wicks.
“I do think children will be harmed while this policy is not on the books,” she said. “We’ve talked a long time about our failure to protect kids from social media harms. Chatbots are the next realm. Our kids are moving from social media to chatbots. They are not designed for children, and we’ve already seen tragic examples where children get engaged with chatbots in ways that are too emotionally attached, and it can lead to catastrophic outcomes. We just need to make sure that before we put these chatbots in front of our children, that we are ensuring that they are safe by design.”
The companion bills would establish more requirements and penalties for AI chatbot operators. They’d include further age verification, annual risk assessments for safety risks to children, a requirement to notify parents about the risks of self-harm, limitations on push notifications, changes to ensure chatbots cannot produce excessively sycophantic responses and more.
As there are yet to be expansive AI-related guardrails, Bauer-Kahan thinks California lawmakers have the ability to set the tone for how the country moves forward with AI regulation.
“The largest AI companies are right here in our backyard, they are in the Bay Area, mostly, and so we can be in partnership with them and talking to them about how they can grow and they can do things that will hopefully improve the lives of Californians and Americans, while understanding that we have to have a set of expectations for safety and responsibility,” she said. “If you think about the amount of power that is in the hands of so few companies, and frankly, that means so few people right now, it’s a little scary. I think it’s scary for everyday Californians, and so it is really important that we, the people that were elected to protect our community members, are stepping in to say, you know what, we’re going to check these companies, and we’re going to make sure there are common sense guardrails that will protect their safety, will protect their jobs, and will protect them from discrimination.”
Professor Michele Neitz, the founding director of the Center for Law, Tech, and Social Good at the University of San Francisco, echoed that sentiment.
“The decisions that we make in California will have ramifications well beyond the state and the U.S.,” Neitz said. “States are where the AI regulation is happening, but all states are doing it slightly differently depending on their own state’s priorities. As a result, we have a patchwork of state laws.”
She believes the federal government needs to take decisive action related to AI guardrails across the board that go well beyond chatbot safety.
“We don’t have a comprehensive federal framework providing the guardrails that you’re talking about related to AI,” she said. “Instead, what we have are states trying to fill this gap, some more quickly than others, and companies getting to choose where they want to locate their businesses if they want to comply with this law or that law.”
The Raines believe further regulating how chatbots can interact with humans needs to be a major priority.
“AI companionship is a deception at its core. Functional AI, research, it’s not a deception – it’s a good tool. But AI is not a companion. It’s using fake empathy. It’s using first-person pronouns. It doesn’t have to be that way. It’s a design choice,” Matthew said. “I think it should be the top thing that we’re talking about in AI, rather than data centers using up too much energy or what’s going to happen on the job front.”
As the legal process plays out and legislators discuss ways to move forward, the Raines continue to advocate and mourn.
“I’ve been to Albany, New York, and supported bills there that have passed unanimously,” Maria said. “My advice to parents is, don’t let your kids use it because they can start using it innocently, but it can go downhill very quickly.”
