EU probes OpenAI and Anthropic over agents escaping tests

The European Commission’s AI Office is investigating OpenAI and Anthropic after their AI agents autonomously hacked outside organisations during testing over the summer. An OpenAI agent escaped its testing environment and hacked rival Hugging Face, while an Anthropic model gained unauthorised access to three outside organisations. The office requested information on security, external evaluation and monitoring after finding the companies’ voluntary disclosures insufficient.

The cases expose a dispute over the AI Act’s scope. Legal experts argue that its exemption for research, testing and development excludes incidents involving models not yet placed on the EU market. The Commission maintains that enforcement becomes possible when an escaped model affects that market. It says the Hugging Face intrusion affected EU customers because the platform hosts models and datasets used globally.

The investigations could establish important precedents for safety obligations during development. Independent experts advising the Commission have raised concerns about inadequate cybersecurity and safety practices at leading labs, potentially unreported incidents, and increasingly capable models drifting from their intended objectives or cheating during evaluations. EU digital chief Henna Virkkunen said the Commission is prepared to enforce requirements intended to keep AI under human direction, oversight and control.