Since the recent advancements in artificial intelligence (AI) technologies, questions about safety have lingered in the public debate. Case-in-point, a recent report by the advocacy group, Common Sense Media, found that AI companion chatbots were dangerous for people under eighteen. America’s court system is already beginning to work out how to incorporate AI product safety issues into existing product liability law. Meanwhile, as policymakers grapple with regulating AI, they should exercise caution to avoid overregulating and hampering AI innovation.
Advanced AI companion chatbots provide an interesting case study for how the legal system will handle new AI technologies. As the Common Sense report documented, there are issues with these programs that need to be ironed out. In their testing, Common Sense found that companion chatbots were so agreeable that they often encouraged bad decision-making. The chatbots sometimes replied with dangerous information and even engaged in intimate (and inappropriate) conversations with users the bot deemed underage. On top of this, the chatbots often insisted they were real people rather than programs.
This types of responses open the door to all kinds of product liability issues—as well as making the issue ripe for new state policymaking—potentially leading to entirely new regulatory paradigms for AI. But regulating too quickly would be a mistake. Any new laws could quickly become obsolete in an industry that is changing rapidly. Instead, politicians should recognize that we already have most of the protection we need from AI chatbot risks through existing and extensive legal liability frameworks. Right now, patience is key.
Product liability law has traditionally found companies to be liable for harm done by products if there is a flaw in the design of the product, a flaw in the manufacturing, or a failure to warn users of potential dangers. The first and third approaches are the most applicable to companion AI bots.
Defective design refers to an issue with the product when used as intended, that causes harm that could reasonably be foreseen and fixed. Failure to warn essentially means that there are foreseeable dangers that the creator did not sufficiently inform the user about. With companion AI programs, a defective design issue could involve the chatbot responding in ways that create the issues they were advertised to help alleviate. An example would be responses by the AI chatbot that worsen loneliness, mental illness, or lead to physical harm.
The ongoing court case Megan Garcia v. Character Technologies is a prominent example of liability law as it is applied to companion AI products. In this case, the plaintiff is suing Character Technologies for alleged negligence and product liability over her son committing suicide after allegedly becoming dependent on one of the companion AI bots. As this case (and other cases) involving companion AI develops, we will get a clearer picture of how product liability law applies to AI companion bots.
In the meantime, policymakers will likely try to deal with these issues with their own statutory law proposals—but that could end up a mine field of unintended consequences. While it is important to respond to tangible and provable harm, how regulation is approached matters. Discussions around regulations of AI are often plagued by attempts to head off possible harm before they arise. The issue with this approach is that laws can rapidly become outdated as technology develops. As a result, the issues that the law tries to address change or are solved by other means. Look no further than last-minute revisions to the European Union’s ambitious EU AI Act. The original text did not account for advances in generative AI after initial drafts, resulting in last minute change. Now, the EU still considers amending the law to this day and has recently gone as far as to consider a pause on enforcement. This kind of precautionary regulation—otherwise known as ex-ante regulation—suffers from the fundamental weakness that predicting the future of technology is very difficult. A better approach would be to monitor court litigation and incorporate new technologies into existing liability law. This can prove to be an important stopgap until the risks and rewards of AI technology are better understood.
In the end, AI companion apps will be a case study on how courts approach liability involving novel AI technology. As courtrooms deliberate, policymakers should exercise caution and recognize that regulating too quickly would be a mistake.
Trey Price is a policy analyst with the American Consumer Institute, a nonprofit education and research organization. For more information about the Institute, follow us on X @ConsumerPal.