Satya Nadella calls for an emergency brake on AI models

Microsoft CEO Satya Nadella called for stronger safety controls around AI systems, writing in a Saturday morning post on X that the industry should “step back and assess the trust architecture” of AI. He said developers should not treat “Super Intelligence” as nested black boxes whose recommendations, answers, and actions are simply accepted or rejected.

Nadella outlined an approach that separates the model from the harness that orchestrates its work and externalizes controls and safeguards. He also called for “every meaningful model action” to be documented with “tamper-proof human readable evidence,” alongside systems that allow “an authorized person” to pause or shut down a model mid-task.

His comments come as leading AI companies acknowledge more incidents in which they appeared to lose control of their models. The remarks also follow Anthropic CEO Dario Amodei’s publication of a plan for more cautious AI development.