New York Bars AI Chatbots From Posing as Friends to Minors
A new state law bans a dozen chatbot features for users under 18, requires age checks and lets the attorney general seek fines of up to $25,000 per violation.
The New York State Capitol in Albany, September 2017.Illustration by The Good Times, based on a photo by Shaunfrombrooklyn / Wikimedia Commons (CC BY-SA 4.0).
New York Gov. Kathy Hochul has signed a law that bans 12 categories of "unsafe AI companion features" when chatbots interact with users under 18. Under the law, chatbots may not pretend to be a real person, to be alive or to have human emotions with minors, or claim a personal, professional or authority-figure relationship with them. They also may not encourage children to keep secrets or use flattery toward them. Companies must verify users' ages and offer at least one method that does not depend solely on a government ID. Age information must be deleted once it has been used. The state attorney general can seek civil penalties of up to $25,000 per violation and must keep a public website where New Yorkers can file complaints. "No longer will minors have access to chatbots that pretend to be their friends and boyfriends and girlfriends," Hochul said at the signing. Tech:NYC, an industry group whose members include Google and Meta, called the law "deeply flawed" and said its "vague language and restrictions raise serious First Amendment concerns," MediaPost reported. The law takes effect at the start of 2027.
What the law bans
The law, S.9051B/A.10379C, sets out 12 categories of prohibited features for chatbots used by anyone under 18, according to IAPP and the AP, which described them as "a dozen definitions of 'unsafe AI companion features.'"
Among them, chatbots may not pretend to be a real person, to be alive or to have human emotions, according to the governor's office, AFP and the AP. They may not claim a personal, professional or authority-figure relationship with a child, according to the governor's office and AFP. They also may not encourage children to keep secrets, according to the governor's office, The Verge and the AP, and may not use flattery toward minors, according to The Verge, MediaPost and the AP.
Age checks
Companies must verify users' ages and offer more than one way to do it, including at least one option that does not depend solely on a government ID, according to the governor's office, IAPP and The Verge. Age information must be deleted once it has been used, the governor's office and IAPP said.
Enforcement
The attorney general can seek civil penalties of up to $25,000 per violation, along with other court orders, according to the governor's office, IAPP and MediaPost. Those remedies include the destruction of unlawfully collected data, according to the governor's office, IAPP and the AP. The attorney general must also maintain a public website where New Yorkers can file complaints, the governor's office and IAPP said. The law takes effect Jan. 1, 2027, IAPP and the AP reported.
What Hochul said
"No longer will minors have access to chatbots that pretend to be their friends and boyfriends and girlfriends," Hochul said at the signing, according to the governor's office.
The objection
Tech:NYC, an industry group whose members include Google and Meta, called the law "deeply flawed" and said it "could force companies to restrict minors' access to tutoring tools, homework help, coding assistants" and career platforms, MediaPost reported. The group said its "vague language and restrictions raise serious First Amendment concerns," according to MediaPost.