AI leaders warn governments over intelligence explosion risk

Geoffrey Hinton and Yoshua Bengio, two researchers widely described as godfathers of modern AI, have joined senior figures from OpenAI and Anthropic in warning that governments should prepare for a possible AI “intelligence explosion”. The concern centers on automated AI research and development, where systems become capable of improving AI without direct human intervention.

The paper is authored by more than 20 people, including Anthropic co-founder Jack Clark and OpenAI chief scientist Jakub Pachocki. It argues that expert-level AI systems working on AI R&D could compress advances that would normally take years into months or less, threatening human control and weakening checks on power across governments, companies and states.

The proposed response includes transparent progress reports on AI-related R&D, independent auditors inside companies, limits on the pace of system improvement, cooperation with datacentres to pause some projects, isolation of automated AI R&D systems, and emergency response plans. Anthropic says AI now produces 80% of its own code, while OpenAI uses autonomous AI agents in areas including model training.

The authors acknowledge uncertainty, including the possibility that supply chains, regulations or AI-assisted safety work could slow or mitigate risks. They nevertheless warn that R&D projects that would take humans months to carry out could be fully automated by AI by 2028.