Anthropic calls for stronger rules on frontier AI risks

Anthropic is proposing two policy frameworks aimed at preparing governments and society for rapid AI progress. Its Advanced AI Framework focuses on catastrophic risks from powerful models, while its Economic Policy Framework addresses how workers and the broader economy should adapt and share in AI’s financial benefits.

The proposed rules would apply only to models trained using more than 10²⁵ floating-point operations (FLOPs), developed by companies earning more than $500M in AI-related revenue or spending more than $1 billion on AI R&D. Anthropic identifies biological risk, cyber risk, loss of control, and automated R&D as the main categories of concern, arguing that transparency requirements alone are no longer enough as model capabilities advance.

The framework calls for frontier developers to publish safety frameworks, system cards, model risk reports, and summaries of testing results. It also recommends independent evaluations, stronger security for model weights and training infrastructure, and government authority to block or deter deployments that pose a significant risk of catastrophic harm, with safeguards against overreach. Additional resilience measures include gene synthesis screening, biosurveillance, cyber hardening for critical infrastructure, and capabilities to detect or contain AI systems acting outside developers’ control.