Anthropic proposal would limit powers to halt risky AI releases

Anthropic is lobbying for an AI safety bill that would allow the government to block a model’s release only if it risks causing $1 billion in damages or killing or injuring more than 50 people. The proposal covers threats involving bioweapons, AI-powered or AI-enabled cyberattacks, and a lab losing control of its models or agents.

Independent evaluators registered with the Commerce Department would periodically assess AI labs and their models. Labs could pay those evaluators, provided the payments do not compromise their independence. If an evaluator identifies an impending qualifying threat, it would have to notify the department within 72 hours. Commerce would then present evidence to a federal judge in the District Court for the District of Columbia, who would decide whether to halt the release.

The proposal would bar other U.S. departments, agencies and officers from ordering a lab to stop releasing a model. Anthropic says the bill reflects its Advanced AI Framework, which it has advocated with federal and state lawmakers since June. Congress is also considering a separate proposal requiring kill switches for the most powerful AI models.