Governor Newsom must soon decide to sign or veto two separate pieces of legislation (AB 1064 and SB 243) regulating AI chatbots. Most troubling is the LEAD for Kids Act (AB 1064), which would prohibit companion chatbots from being made available to kids unless the chatbots are “not foreseeably capable” of encouraging harm, such as self-harm, substance abuse, eating disorders, or offering unlicensed mental health therapy. That sounds nice in theory, but in practice, AB1064 will lead to numerous unintended consequences: legally, practically, and ethically.
Let’s start with the many technical issues. Everybody is on the side of protecting kids—but the LEAD for Kids Act is chock-full of overly broad definitions and vague requirements. It limits the applicability of the law to those under 18, which means companies will need to verify the ages of all users, raising many of the same consumer privacy and security concerns central to state age verification law debates. It is also the latest example in a long line of California technology regulations that are likely to be unconstitutional in that all of them pressure or mandate businesses to “censor speech based on its content.”
All those reasons are wonkish reasons to avoid this specific policy—the legislative details do matter—but above all else, the policy itself is misguided in its effects. The law would likely push affordable and widely accessible mental healthcare services out of the reach of Californians struggling with mental health issues.
Not only would specific licensing requirements increase costs, but companies would likely respond to the law by withdrawing mental healthcare services, especially in difficult age verification cases. The law would also encourage companies to err on the side of providing fewer service offerings, unless they risk a punitive $25,000 per-violation penalty while unleashing the plaintiff’s bar in a series of civil lawsuits.
California is being too quick to ban a service that is still being studied and has been found to be beneficial to teens and adults in many cases. Some studies have found that AI companion chatbots, which focus on establishing a connection with people, reduce loneliness. Others have found that conversational AI is a cheap alternative compared to traditional therapy, and improves mild depression among teens, and it can yield positive benefits. The reality is that many teens use chatbots for emotional and mental health support in productive and positive ways.
All of this is bad news for the roughly two-thirds of Americans with a diagnosed mental health condition who cannot access treatment or healthcare. More than half of U.S. counties do not have access to a mental health professional. AB 1064 is bad news, especially in California, where the number of minors who received mental health treatment has declined since 2010, and as of 2021, less than a quarter of the population’s mental health needs were met.
As a general matter, mental healthcare is expensive. The average cost for a therapy session in the U.S. is $140, but the average for California is $162. In a state where the average cost of accessing therapy is above the national average, low-cost AI chatbots offer alternative solutions at a fraction of the cost. Consider ChatGPT or other more targeted services for mental health that only cost $20 per month with unlimited availability. AB1064 would take away this imperfect option even as AI models continue to improve over time.
Lawmakers also should not forget that California is experiencing a mental health care professional shortage, with 23 out of 58 counties having fewer than 1 psychiatrist per 10,000 residents and 6 counties having none at all. This shortage makes finding a mental healthcare professional—let alone a trusted professional available at all hours—extremely difficult. It’s no wonder then that California ranks 38th in children’s mental health care access and 33rd in youth mental health.
Rather than legislating from a place of fear, California policymakers should have recognized that the mental healthcare outlook in the state is in many ways already bleak and that technological improvements offer an opportunity to make AI mental health services safer, more accessible, and more affordable. Unfortunately, that’s not what’s happening.
Utah provides a better approach to AI therapist regulation by focusing on a balanced approach to consumer protection, transparency, and data protection. Utah—recognizing the risks and rewards of AI therapy—passed a law earlier this year that requires mental healthcare providers’ involvement in the development and review process of these chatbots. It also requires preliminary testing to make sure these models do not pose a significant consumer risk. To evaluate that risk, AI therapy chatbots are evaluated by comparing their output to clinical best practices and ensuring they present no greater risk than a licensed mental health therapist.
Unlike California, the harms are specific, targeting its definitions to AI therapy chatbots, which explicitly exclude other general-purpose chatbots. They also compare chatbots to the world as it exists today, rather than a utopian vision of the world that does not exist. The approach is also balanced and careful in not overreaching to other AI technologies.
Above all, Utah’s approach sees the benefits AI chatbots can provide, and the extent to which people are using these services. However, it sets specific guardrails for the harms they may bring. This type of regulation doesn’t result in immediate bans or hefty penalties, as California does. It allows technology to keep improving. This approach better balances accountability and innovation—and should be the kind of policy California considers.
As Governor Newsom decides to sign or veto this legislation, he should consider whether Californians are made better or worse off under a law that shrinks access to mental healthcare tools. Mental healthcare tools will continue to improve—and acting now would be premature. After all, the foundational rule of medicine is to “first, do no harm.” In its current form, the LEAD for Kids Act does more harm than good.
Logan Kolas is the Director of Technology Policy and Pablo Garcia Quint is a policy analyst at the American Consumer Institute, a nonprofit education and research organization. For more information about the Institute, follow us on X @ConsumerPal.