New York’s governor recently signed an AI chatbot bill that has been on her desk since June. The law will prohibit AI chatbots from using features considered unsafe for minors and focuses on “covered AI companions.” In other words, genAI chatbots that give ongoing and adaptive responses to user queries. The law will go into effect on January 1, 2027.
The law applies to operators of covered AI companions and has narrow exceptions. These include chatbots used solely for customer service that cannot respond to topics outside of the customer service purpose. It also applies to tools that are used only for efficiency improvements, research, or technical assistance. Again, provided that the tool cannot respond to topics outside that purpose. It also does not apply to internal business or employee productivity tools.
Under the law, operators will not be allowed to provide specifically listed unsafe AI companion features to users whom the operators have actual knowledge are under 18. Related to this, companies will need to ensure a user isn’t a minor before offering those features. To determine age, companies must use “reasonable” methods that, among other things, stop someone from trying to circumvent the process. They must also offer more than one method and not rely only on self-declarations of age.
Features that cannot be offered to minors include chatbots that pretend to be human, claim to have a personal relationship with the user, or act as an authority figure. The law also bans certain problematic outputs, like those that promote or facilitate suicide. AI companions for minors also cannot include design features that simulate companionship or an interpersonal relationship with a user.
The law gives the Attorney General rulemaking authority. Among other things, rules may be promulgated for age assurance mechanisms or other features that should be prohibited. There is no private right of action. Instead, enforcement rests with the state Attorney General. The AG also has general rulemaking authority and enforcement power. . The law provides, among other things, for injunctive relief and civil penalties of up to $25,000 per violation. Contractual clauses that try to waive liability or shift it to users will be considered void as a matter of public policy.
Putting it into Practice: These laws have been passed with regular frequency (including in Georgia, Oregon, and Washington). Companies that offer covered AI chatbots will want to review the tools functionality, especially if they could be used by minors in New York. Reviews should include how age assurance will be conducted.