Understanding Sycophantic AI systems: A legal and regulatory lens – Part 2

As sycophantic AI systems blur the line between assistance and manipulation, regulators and courts should determine whether flattery by design constitutes deception by law.
Understanding Sycophantic AI systems: A legal and regulatory lens – Part 2
Published on

EVOLVING BEHAVIOURAL DIMENSIONS of Artificial Intelligence (‘AI’) systems are exerting fresh challenges for existing laws, rules, and governance. While laws targeting chatbots are missing in most places, different kinds of regulations still apply. As technology grows, old rules might handle new tech problems. Illustratively, if fake praise from a robot crosses into slander, misrepresentation, or trickery. Then regular court claims like harm suits or crime charges step in. Sycophancy in AI systems can also lead to allegations of negligence or cheating. When AI chatbots push false claims, say, about health treatments, it may run afoul of rules meant to shield patients from medical misdirection.

A first look at the existing frameworks

One path for legal pushback comes from laws that address questionable business conduct. Across the United States of America, the Federal Trade Commission already polices deception in commerce. An Executive Order (‘EO’) issued by President Trump in late 2025 gives the relevant executive agency fresh direction to sort out how its power stretches into AI replies. If AI systems are found to misinform, authorities can initiate actions under the same framework meant for misleading ads or scams. Section 6 of the EO mandates agencies to weigh model disclosures, while Section 7 focuses on awareness around misleading results. 

When AI chatbots push people toward harmful actions, defining liability becomes a difficult task. Right now, courts in the US frequently protect online spaces using Section 230 of the Communications Act, 1934, enacted as part of the Communications Decency Act, 1996, which guards them against lawsuits over content users post. The provision provides limited federal immunity to providers and users of interactive computer services. Yet, uncertainty hangs over whether that immunity extends to content generated by AI instead of humans. 

Behind closed doors, tech firms and deployers have been inserting warnings and disclosures such as ’not professional guidance’ to distance themselves from harm. Rules around coverage, permission, and protection for AI systems remain underdeveloped.

Worldwide, officials are shaping rules focused squarely on how artificial intelligence acts. In Europe, a Regulation (EU) 2024/1689 of the European Parliament, and of the Council, colloquially called the ‘AI Act’ is coming into force in stages. It places strict responsibilities on AI deemed highly risky. One key pillar of the regulation is to counter and address systems designed to mislead or influence people unfairly. Specifically, Article 5(1)(a) of the AI Act outlaws AI using methods that twist someone's choices in serious ways - when those shifts lead to damage. AI models that lean into human preferences might fall under this rule. Providers of certain AI systems prone to sycophantic behaviour should maintain safeguards to identify and mitigate associated risks, preserve opportunities for meaningful human intervention, and ensure transparency about the AI nature of the interaction, including clear disclosure when a user is communicating with software.

Across the globe, policy experts are looking at rules to ensure AI systems reveal why they gave certain suggestions to enable greater transparency.
Loading content, please wait...
The Leaflet
theleaflet.in