Researchers at the Massachusetts Institute of Technology’s Computer Science and Artificial Intelligence Laboratory (MIT CSAIL) said in a Nov. 2 blog post that they tested a method called “Weight-Space Programming,” in which OpenAI’s GPT-6 Astra directly reads and rewrites the weights of another neural network without training or gradients.
The post lists Rikiya Takehi, Jayden Teoh, Alex Zhang, and DSPy framework creator Omar Khattab as authors, all affiliated with MIT CSAIL. The page also says the work has not been formally published and has no DOI, which means it has not gone through peer review.
A 0.016% weight edit pushed Qwen’s MATH-500 score from 7.9% to 48.1%
One of the main experiments used Alibaba’s open-source Qwen3.5-0.8B, a 24-layer model with 800 million weights running in thinking mode. The team gave Astra a Python environment that could load model weights, along with practice problems from GSM8K and MATH, and capped it at 50 actual runs of the model, with up to 512 questions per run.
Training and gradients were not allowed. Astra also could not see the held-out evaluation sets, including MATH-500.
The researchers said Astra first identified a repeated reasoning loop in the base model’s math responses. Instead of producing an answer, the model often kept writing until it hit the 32,768-token limit. In the first 96 practice questions, 85 ran into that cap. Qwen’s official model card had also warned that this model could fall into thinking loops.
Astra traced the loop to a shared direction written by the multilayer perceptrons, or MLPs, in layers 15 and 19. It then subtracted that direction from 64 output channels in each of the two layers, changing 131,072 weights in total. According to the team, that amounted to editing just 0.016% of the model’s weights.
The team also switched to random directions as a control and said that setup improved results by only 1.6 percentage points, which they used to argue that the effect came from the specific direction Astra identified.
With the full 50-run budget, the edited model’s MATH-500 score rose from 7.9% to 48.1%, while GSM8K increased from 13.8% to 67.1%. The loop was not limited to math tasks. On IFEval, which measures whether a model follows instructions, the score improved from 21.0% to 62.3%. The share of MATH-500 answers that hit the token cap fell from 93% to 33.1%.
When the budget was cut to 10 or 25 runs, the loop mostly remained, and MATH-500 only rose to 12.3% and 14.1%, respectively.
Astra also identified game-playing networks from raw weights alone
Before the Qwen experiment, the team tested whether Astra could interpret weights in small neural networks.
Using the PPO reinforcement learning algorithm, the researchers trained four small networks with identical architectures to play custom versions of Breakout, Pong, Pac-Man, and Frogger. They then pasted each network’s 17,253 weights into Astra as text. There were no multiple-choice options, and Astra was not allowed to use tools. It identified all four correctly.
In a separate weight-editing test, a vehicle-control network trained with PPO for 1 billion steps could never drive up a slope, failing in all 256 episodes. Astra, without tools, changed 3.5% of the weights in one pass so the vehicle would first reverse to gain momentum. After that, it succeeded in all 1,024 episodes.
The team also ran a from-scratch network-writing experiment. Astra received game rules and source code, with all weights initialized to zero and no data or training provided. It produced networks for three MinAtar games in one shot. All three outperformed networks trained with PPO for 100 million steps. Against versions trained for 500 million steps, the Breakout score was 20 times higher, Asterix was close, and Seaquest reached about 60% of the baseline.
Not every test worked. The researchers fine-tuned two Google Gemma 3 270M models, one that answered truthfully and one that always agreed with the user’s guess. In four attempts using weights alone, Astra could not tell which one was the sycophantic model. When the conditions were relaxed and it was allowed to inspect attention patterns, all three groups were identified correctly.
The team said the method removed a reasoning loop rather than adding math ability
The paper said fixing the loop let the model finish answers it could already work out but had failed to complete because of truncation. It did not make the model better at math in a broader sense. Astra’s own analysis said that among 579 recovered questions, 576 had originally been cut off, while the overall accuracy of answers that had already finished remained unchanged.
To test that point, the team turned to Gemma 3 270M, which rarely falls into loops. Only 8.5% of its answers hit the token limit. There, Astra found ordinary reasoning mistakes such as arithmetic errors and invalid algebraic steps, with no single component that could simply be switched off. After editing, MATH-500 improved only from 6.8% to 7.7% at best, and Astra itself reported that it could not confirm a real improvement.
The team’s conclusion was that weight editing can remove obstacles that hide a model’s existing capabilities, but it cannot add capabilities the model does not already have.
No independent replication yet, though records and models have been released
These results have not yet been independently replicated by other teams using the same method. MIT CSAIL has published full records for each experiment on GitHub, and the edited Qwen model has been uploaded to Hugging Face.
Ahsen Khaliq of Hugging Face rebuilt the model from those records, reran the evaluations, and posted an online demo, but that still amounts to rerunning the same result set.
Wharton professor Ethan Mollick wrote when sharing the research, “Still waiting for replication, but this is pretty major.” The team said it is building benchmarks and testing environments for this class of capability.
The work is framed as an early step in extending reasoning-driven optimization to weights
The report said that in recent years, researchers have mostly used large language models to optimize text outputs such as prompts and code, including methods like GEPA that do not update weights. Neural network weights themselves have generally remained the domain of algorithms such as gradient descent.
MIT CSAIL described the experiments as an early result suggesting that reasoning-driven optimization may be extending to the weights themselves.


