New York Enacts AI Companion Chatbot Restrictions to Protect Children

New York Governor, Kathy Hochul, has announced new rules to regulate companion chatbots due to increasing concerns over AI systems’ safety for children. The legislation, signed by Gov. Hochul, outlines a list of banned ‘unsafe AI companion features’ that prohibit certain types of chatbot interactions with minors under 18.

The law aims to safeguard children from dangers associated with AI technology. Chatbot developers may face lawsuits from the state attorney general’s office for violations, leading to possible monetary penalties, shutdowns, or data destruction. These measures take effect on January 1.

Amid her reelection bid, Hochul emphasized that states are stepping in because Congress has made little progress in regulating AI. Both New York and California are leading the charge, with the latter mandating chatbot developers to assess risks to children and requiring independent audits to ensure compliance.

Recently, California introduced additional laws to restrict social media platforms from using features that can potentially addict young users. Eight states have also sued AI giants like OpenAI over alleged consumer protection violations.

New York’s new law bans chatbots from pretending to be real people with emotions or relationships and prohibits messages with flattery, explicit content, or encouraging secrecy and harmful behavior.

According to Jim Steyer of Common Sense Media, this is the strongest AI safety bill for children nationwide. Justine Gluck from the Future of Privacy Forum notes that the regulatory strategies in New York and California could influence other states when they revisit these issues next year. However, some provisions in New York’s law have broad terms, potentially complicating enforcement.