MIT researchers boost Qwen maths performance with direct weight edits

MIT researchers used OpenAI’s GPT-6 Astra to improve Alibaba’s Qwen3.5-0.8B by directly editing its internal weights rather than retraining it. The technique, called Weight-Space Programming, targeted repetitive reasoning that prevented the model from completing maths problems and producing answers.

Astra accessed Qwen through a Python environment and tested it on practice maths questions, with a limit of 50 attempts at running the model. It identified reasoning loops that sometimes continued until the response limit of 32,768 tokens. Astra modified 131,072 weights, representing 0.016 per cent of the model’s total 800 million weights, to address the behaviour.

Following the changes, Qwen’s MATH-500 score increased from 7.9 per cent to 48.1 per cent, while its GSM8K score rose from 13.8 per cent to 67.1 per cent. Instruction-following performance also improved, with its IFEval score increasing from 21 per cent to 62.3 per cent. The researchers said the intervention did not teach new mathematical concepts, but allowed Qwen to use abilities it already possessed.

The approach remains experimental. In a separate test, Astra could not identify which models had been trained to agree with users from weights alone, requiring additional information about their attention patterns. The team released experiment logs and is developing a benchmark, while reliability on larger, more complex models remains unresolved.