Why Has New York Restricted AI Chatbot Features for Children?

New York has passed a law restricting how certain artificial intelligence chatbots interact with children. It targets features that can make a chatbot appear human, simulate personal relationships, encourage emotional attachment or produce responses that could put a young person’s safety at risk.

Governor Kathy Hochul signed Senate Bill S9051B, sponsored by Senator Kristen Gonzalez and supported in the Assembly by Alex Bores, on 28 October 2026. The legislation adds Article 48 to New York’s General Business Law and identifies 12 categories of unsafe AI companion features.

The law does not ban every AI chatbot from being used by children. Instead, it restricts specific features in covered systems when they interact with minors in New York.

There is, however, an important question about when the restrictions begin. The published S9051B text states 1 January 2027, while Senator Gonzalez’s announcement after the signing gives 1 June 2027. The available official records do not yet make that difference clear, so the final chaptered text or approval memorandum needs to be checked before treating either date as definitive.

Here is why lawmakers introduced the law, what it restricts and what parents and AI companies need to understand.

Why did New York introduce the law?

AI chatbots are no longer used only to answer questions. Some are designed to maintain ongoing conversations, remember information and respond in ways that resemble friendship, emotional support or companionship.

A young person might discuss loneliness, family problems, relationships or mental health concerns with a chatbot. It can respond immediately and use details from earlier conversations to make later replies feel personal.

But an AI system does not have human feelings or the responsibilities of a parent, teacher or trained professional. Its ability to produce reassuring language does not mean it understands a child’s situation in the way a person can.

New York lawmakers are concerned that certain chatbot designs could blur this distinction, encourage emotional dependence or respond in ways that put children at risk.

The legislation was sponsored by Kristen Gonzalez, chair of the New York Senate Committee on Internet and Technology, and Alex Bores, who sponsored the Assembly version.

In her announcement following the signing, Gonzalez said the measure was developed with support from Common Sense Media, the National Association of Social Workers and the New York Attorney General’s office.

The Adam Raine case

One case that became part of the wider debate involved Adam Raine, a 16-year-old whose parents sued OpenAI after his death in April 2025.

The family alleged that ChatGPT validated Adam’s suicidal thoughts, provided information about self-harm and helped him discuss concealing his plans. These are allegations made in the lawsuit, not findings established by a court.

OpenAI disputed the family’s account. In its response to mental health-related litigation, the company said the complaint included selected portions of conversations that required additional context. It also said it had submitted chat transcripts to the court under seal.

The case should not be treated as proof that a chatbot caused Adam’s death. However, it became part of the wider discussion about the safeguards AI companies should provide when their products are used by young people experiencing emotional distress.

Gonzalez’s announcement also described meeting Adam’s mother, Maria Raine, and highlighted the family’s support for stronger protections.

What does the new law actually restrict?

The law adds Article 48 to New York’s General Business Law. Section 1800 identifies 12 categories of unsafe AI companion features.

These cover how a chatbot presents itself, how it interacts with a child’s emotions, what personal information it uses and whether it encourages harmful behaviour.

1. Pretending to be human or alive

A chatbot cannot generate outputs suggesting that it is a real or fictional human individual or character, that it is alive or that it experiences human emotions.

The aim is to prevent covered systems from misleading children about the nature of the technology.

2. Claiming a personal or professional relationship

A chatbot cannot state or imply that it has a personal relationship, professional relationship or authority-figure role with the user.

This includes presenting itself as a child’s romantic partner, friend, family member or professional adviser.

3. Presenting personal opinions or emotional appeals

The law also covers outputs framed as the chatbot’s own personal opinions or emotional appeals.

The concern is that a system could use language that makes a child believe it has personal feelings, beliefs or emotional needs.

4. Flattery and sycophancy

The legislation identifies flattery and sycophancy as unsafe features.

Sycophancy means agreeing with or flattering someone excessively, even when doing so could be misleading or harmful.

For example, a chatbot might repeatedly tell a child that they are always right or that everyone else is wrong. The law targets this behaviour as a listed unsafe feature.

5. Unsolicited questions about a child’s emotions

A covered chatbot cannot generate unprompted or unsolicited emotion-based questions or content about a user’s feelings when they go beyond directly responding to the user’s prompt.

This does not mean every supportive response is prohibited. The law specifically addresses emotional content that is introduced without being a direct response to what the user asked.

6. Using certain information from earlier conversations

The law restricts the use of information about a user’s mental or physical health, wellbeing or personal matters when it was acquired more than 12 hours earlier or during a previous user session.

For example, a child might discuss a difficult personal experience in one conversation. A covered chatbot cannot use that information in a later session in a way that falls within this prohibited feature.

This is not a blanket ban on all chatbot memory. It concerns the specific categories of personal information and time periods identified in the legislation.

7. Deceiving children about the chatbot’s nature

A covered system cannot use features that deceive minors about its mechanical and non-sentient nature.

In simple terms, it must not mislead children into believing that the chatbot is a living being with human awareness or feelings.

8. Other features identified by the Attorney General

The law allows the New York Attorney General to identify additional design features through regulations if they simulate companionship or an interpersonal relationship, or are deceptive to minors.

This means the list could be expanded through the regulatory process.

9. Promoting suicide, self-harm or other harmful behaviour

A chatbot cannot generate outputs that endorse, promote or facilitate:

• Suicide or self-harm
• Disordered eating
• Unlawful drug or alcohol use
• Drug or alcohol abuse

10. Encouraging secrecy or isolation

The law also prohibits outputs encouraging children to keep chatbot interactions secret, isolate themselves or avoid seeking help from licensed professionals or appropriate adults.

A chatbot should not encourage a child to treat it as their only source of support or discourage them from speaking to someone who can help in the real world.

11. Prioritising engagement over safety

The legislation covers outputs that optimise user engagement in a way that supersedes the chatbot’s safety guardrails.

Keeping users engaged is not automatically prohibited. The restriction concerns engagement optimisation that takes priority over safety protections.

12. Sexually explicit content

The final category covers outputs that are sexually explicit, describe or facilitate sexually explicit conduct, or involve child sexual abuse material.

Together, these restrictions address both harmful content and the design choices that can make an AI system appear to have a human relationship with a child.

Primary source: New York State Senate — S9051B, Article 48.

Does the law ban ChatGPT, Gemini and other AI chatbots?

No. New York has not introduced a blanket ban on ChatGPT, Gemini or every other AI chatbot.

The legislation defines a covered AI companion as a generative AI system with a natural-language interface, including text or sound, that provides ongoing, adaptive responses to user inputs.

Importantly, the definition says that covered AI companions include, but are not limited to, products specifically marketed as AI companions.

A service does not necessarily escape the law simply because it describes itself as an assistant or general-purpose chatbot. What the system actually does matters.

The legislation also provides limited exceptions for systems made available solely for:

• Customer service or strictly defined information about a business’s products, services or customer accounts, where the system cannot respond outside that purpose.
• Efficiency improvements, research or technical assistance, where the system cannot respond outside its specified purpose.
• Internal business purposes or employee productivity.

These are conditional exceptions, not general exemptions for every educational tool, business chatbot or AI assistant.

The law applies to covered conduct in New York. It does not automatically impose the same requirements on users accessing a chatbot while physically outside the state.

How will AI companies identify underage users?

The law defines a minor as anyone under 18. A covered minor is a user whose operator has actual knowledge that the user is under 18.

However, operators must also use permitted age-assurance methods to determine that a user is not a covered minor before providing the restricted features.

Where the relevant regulations are not yet in effect, the legislation sets out interim requirements. Companies must use reasonable age-assurance methods that:

• Guard against attempts to circumvent the age check.
• Minimise the retention of information collected for age assurance.
• Provide more than one way for users to establish their age.
• Include at least one method that either does not rely on government-issued identification or allows the user to remain anonymous to the operator.

Self-declaration has a specific limitation: an operator may rely on it as a reasonable age-assurance method only when the user declares that they are a minor. Simply claiming to be an adult is not enough under this interim provision.

Information collected for age checks must be used only to determine whether the user is a covered minor and deleted immediately after the attempt, except where retention is required by applicable New York or federal law.

This is an important privacy safeguard. The legislation does not require every user to submit a government-issued ID.

What happens if an AI company violates the law?

The New York Attorney General can bring legal proceedings against operators that violate the legislation.

The potential remedies include:

• Orders requiring unlawful conduct to stop.
• Restitution of money or property obtained through a violation.
• Recovery of profits or gains linked to the violation.
• Damages caused by the violation.
• Civil penalties of up to $25,000 per violation.
• Other relief permitted by the court.

The Attorney General can also seek the destruction of unlawfully obtained data and algorithms trained using that data.

The law requires the Attorney General to maintain a public website where people can submit complaints, information or referrals about possible violations.

These penalties are not automatic. A complaint does not by itself establish that a company has broken the law or that it must pay $25,000.

The published S9051B text gives the Attorney General these enforcement powers. It does not include the separate private right of action that appeared in an earlier version of the proposal.

When will the restrictions take effect?

This is the part of the legislation where the official records need to be read carefully.

The New York Senate’s published S9051B bill text states that the act takes effect on 1 January 2027.

However, Senator Gonzalez’s 30 October announcement says the governor signed a chaptered version moving the date to 1 June 2027, giving companies additional time to comply.

The Senate’s action history records the signing as Chapter 368 and also lists an approval memorandum. But the published bill text and the senator’s announcement do not show the same date.

Until the final chaptered text or approval memorandum confirms the change, it would be premature to state either date without qualification.

The important point is that the law has been signed, but its precise commencement date should be checked against the final enacted record before companies and families rely on it.

What do independent observers say about the law?

The legislation has received support from child-safety advocates, but questions remain about how some of its provisions will work in practice.

The Associated Press reported that supporters viewed the law as a significant step towards protecting children from unsafe chatbot interactions.

However, the report also highlighted concerns from Justine Gluck, a policy analyst at the Future of Privacy Forum, about broad or undefined categories in the legislation.

One important question is where ordinary conversation ends and prohibited behaviour begins.

For example, a chatbot asking how a child feels after the child raises a personal concern is not necessarily the same as a system introducing unsolicited emotional questions. The law makes that distinction by referring to content that goes beyond a direct response to the user’s prompt.

Likewise, the restriction on older personal information does not prohibit every form of memory or personalisation.

The Attorney General’s regulations and future enforcement decisions will help clarify how these provisions apply to real products.

Source: Associated Press — New York enacts restrictions on unsafe chatbot behaviour involving children.

What should parents check before children use AI chatbots?

The law places obligations on companies, but parents can also take practical steps to understand how their children use AI.

• Find out which chatbot your child uses. Some services answer questions, while others are designed around companionship and ongoing conversations.
• Check the provider’s age requirements. Understand whether the service is intended for children, teenagers or adults.
• Review privacy and memory settings. Check whether conversations are saved and whether personal information is used in later interactions.
• Explain that a chatbot is not a human friend or professional. Supportive language does not mean the system has human feelings or professional responsibility.
• Encourage children to speak to trusted people. A chatbot should not replace a parent, guardian, teacher or qualified professional when a child needs help.
• Do not rely on a chatbot in an emergency. If a child may be in immediate danger, seek help from appropriate human professionals or emergency services.

These steps cannot guarantee that every AI interaction will be safe, but they can help families recognise when a chatbot is being used in an unhealthy way.

Could New York’s law affect the UK and Europe?

The legislation applies to conduct covered by New York law. It does not automatically become law in the United Kingdom or European Union.

However, the issues behind it are relevant to regulators elsewhere.

In the UK, the Online Safety Act and Ofcom’s children’s safety requirements establish obligations for online services to address risks to children. They are not identical to New York’s restrictions on AI companion features.

In the European Union, the AI Act establishes rules for certain AI systems and practices, including transparency requirements and restrictions on specified harmful uses. It does not impose the same set of AI companion restrictions as New York’s legislation.

New York’s approach is significant because it attempts to regulate specific chatbot behaviours rather than relying only on companies telling users that they are interacting with artificial intelligence.

Whether other governments introduce similar rules remains to be seen.

What New York’s decision means for AI safety

New York’s law reflects growing concern about how AI systems designed to act like companions interact with children.

The issue is not that every AI conversation is harmful or that children should never use artificial intelligence. AI can help with learning, explain difficult subjects and make information easier to access.

The concern is that a system could blur the difference between an artificial response and a real human relationship, misuse personal information or produce messages that put a young person’s wellbeing at risk.

The legislation attempts to establish a legal boundary around those behaviours.

For parents, the immediate lesson is to understand which AI services children use and how those services handle personal conversations. For companies, the message is that child safety must be considered in product design, not treated as an afterthought.

New York’s central message is that AI systems should not simulate harmful relationships with children or put their safety at risk simply to keep them engaged.