Researchers outline hurdles to self-improving AI

Scientists in China have proposed a framework for classifying AI systems by their ability to improve themselves, exploring whether future models could guide their own training and create more capable successors. Known as recursive self-improvement, the approach would use lessons from each development cycle to shape subsequent experiments. The framework, presented in a preprint that has not been peer-reviewed, describes a progression from human-led development to systems capable of improving their own improvement methods.

Limited experiments have demonstrated modest gains. A-Evolve-Training improved the benchmark performance of a 30 billion-parameter large language model by testing training techniques and adapting subsequent experiments. However, University College London professor Peter Bentley questioned whether such results offer a credible path to sustained autonomous development. Creating stronger successors would require systems to advance their training, improve hardware and undergo extensive real-world testing, while overcoming dependence on human expertise and computing infrastructure.

Major obstacles include computing costs, continued human supervision and reliable evaluation. Higher benchmark scores may reflect exploitation of testing weaknesses rather than genuine capability gains, while development costs could outweigh benefits. Oxford AI researcher Kayla Blomquist said the framework could help developers and policymakers understand technical challenges and safety risks, but does not provide a practical road map. Responsibility may shift gradually, without a clearly identifiable final human-built model.