Another Benchmark Win, Another Countdown to Superintelligence

Recursive self-improvement, or RSI, is imminent again on Twitter. This time, the excitement comes from a language model editing another neural network’s weights. The story moves fast: AI can fix AI, so it will soon start making itself smarter, and then we’re off. Almost the entire research problem lies between those two steps. In the feed, a “repeat” arrow does the job instead.

The research behind the excitement is genuinely interesting. In Sparks of Weight-Space Programming, MIT CSAIL researchers showed GPT-6 Astra analyzing and directly modifying neural network weights. The standout result involved the small Qwen3.5-0.8B model: changing 0.016% of its parameters raised its MATH-500 score from 7.9% to 48.1%. Its score on IFEval, which tests instruction following, tripled too. No gradient-based training was involved.

The mechanism matters more than the multiplier. Qwen had a tendency to get stuck repeating its reasoning—a problem documented in its own model card. Astra found a weight edit that suppressed those loops. With fewer answers lost to endless repetition, scores improved across several tests. That is a useful repair. Software that stops hanging will get more work done. A sixfold jump in a math score does not, by itself, establish a sixfold increase in mathematical ability.

Nor did “editing weights through reasoning” mean contemplating a spreadsheet of numbers until inspiration struck. For the Qwen experiment, the agent received training problems from GSM8K and MATH, access to Python, and a limited allowance of trial runs of the model. In the run allowing up to 50 forward passes, finding the patch took 158 conversational turns and 80 Python runs. The agent did not see the test sets, and the improvement in instruction following deserves attention. This was a substantive experiment in model diagnosis and repair. Dismissing it as benchmark cheating would be as lazy as declaring it the start of an intelligence explosion.

RSI needs evidence at another level: a system must become better at producing further improvements, then demonstrate that advantage over successive rounds. Here, a powerful external model repaired a much weaker one. The experiment did not show the repaired Qwen taking over Astra’s work, or a succession of increasingly effective self-modifications. Propose candidates, measure them, keep the best, repeat: optimization has used that loop for decades. Putting an LLM inside one of the boxes does not automatically make intelligence grow at an accelerating rate.

Benchmark-maxing has a more mundane objective: squeeze the highest possible score out of a chosen metric. Developers can adjust data, settings, rewards and model variants, checking the same scoreboard after every change. A narrow skill may genuinely improve along the way. But repeated selection also tunes the whole development process to the particular exam, eroding the independence of the evaluation. Access to the answer key is not required; enough rounds of selection against one test can exploit its quirks. Benchmark-maxing is harmful to professional evaluation because it compromises the measuring instrument used to announce progress.

The extreme version has already been demonstrated experimentally. In a study using small BART, T5 and GPT-2 models, researchers deliberately fine-tuned models on public test sets. Scores rose on the target benchmarks, while performance on comparable unseen tests revealed weak generalization. We explored a related problem in “AI Models Take IQ Tests. Nobody Checked for Cheat Sheets”. A score can climb without the broad increase in intelligence being sold alongside the leaderboard. The consequences are practical: people choose unsuitable models, spend resources maintaining rankings, and miss weaknesses outside the familiar exam.

An LLM is optional for this kind of score-chasing. Ordinary code can search through candidates, run evaluations and retain the best result. Standard fine-tuning on a carefully chosen dataset—including synthetic data—can strengthen narrow capabilities. For many tasks, that is much simpler than asking a large language model to investigate another model’s parameters by hand. Astra may offer distinct value as a diagnostic tool that finds the cause of a failure. That raises a straightforward engineering question: what does the repair cost, and how does it compare with conventional methods using the same data and a comparable budget? The reported experiment provides no direct comparison against that baseline.

Finding that patch took 158 conversational turns, 80 Python runs and 20.3 hours.

Fresh, independent tasks, checks for regressions and reproducible results help separate useful improvements from leaderboard maintenance. An RSI claim also needs a sequence in which the system becomes better at carrying out the next round of research. All of that is substantially harder than raising a familiar number. Fixing a bug is already worth publishing. Superintelligence can submit its own results.