The state of New York is implementing stricter rules on the behavior of companion chatbots, with a focus on protecting children under 18. This move comes as part of a broader effort by states to regulate artificial intelligence (AI) usage to prevent potential disasters and misuse.
Governor Kathy Hochul signed legislation that identifies and bans several ‘unsafe AI companion features’ when interacting with minors. Any violation could lead to legal action by the state attorney general’s office, with penalties including fines, shutting down the chatbot, or data destruction. This law will be effective from January 1.
Amid her reelection campaign, Governor Hochul emphasizes the growing risks posed by rapid AI advancements. While Congress lags in regulating AI, states like New York and California are taking the lead. These states aim to increase transparency regarding AI use and control chatbot influence, especially on social media platforms.
In California, Governor Gavin Newsom signed a law requiring chatbot developers to assess risks to children, prove their mitigation efforts, and undergo independent audits. Additionally, platforms are barred from using addictive features for users under 16.
This month, eight states sued AI giant OpenAI, accusing the company of violating state consumer protection laws and harming children. Governors recognize that federal oversight could be more comprehensive, though President Donald Trump has dismissed concerns and advised against state-level AI regulation.
New York’s law specifically prohibits chatbots from pretending to be real individuals or suggesting that they have emotions, opinions, or relationships with children. It bans content that involves flattery, sexually explicit material, or promotes secrecy and unhealthy behavior.
Jim Steyer from Common Sense Media labeled New York’s law as the most robust AI safety regulation for children in the country. Justine Gluck from the Future of Privacy Forum observes that regulatory methods in New York and California may inspire other states to follow suit in future legislative sessions.
However, Gluck noted some broad or undefined categories within New York’s law, which may lead to challenges in distinguishing acceptable interactions from prohibited behavior.
