When engineers build a neural network today, its weights - the long lists of numbers that store everything the network has learned - are written by optimization algorithms, not by people. A new report from Rikiya Takehi, Jayden Teoh, and Alex Zhang at MIT CSAIL suggests this may be starting to change. They show that a frontier language model, GPT-6 Astra, can read the raw weights of a network, reason about what they compute, and edit or even write them from scratch. The authors call this ability weight-space programming.
In reading tests, the team handed Astra the 17,253 weights of a small network trained to play one of several simple grid games, with no game named. Astra correctly identified each game, for example spotting a bottom paddle and rows of bricks in the weights of a Breakout player. It also told what shapes small image generators produce and, given only the middle layers of three small language models, named the language each was trained on and located every copying head, the components that let a model repeat text seen earlier, also known as induction heads, matching measurements exactly.
From reading weights to writing them
The team then asked Astra to modify weights. Given a car control network that always got stuck at a ramp, it edited 3.5 percent of the weights and raised the success rate from zero to 100 percent. Asked to make the small language model Qwen3.5-0.8B better at math without any training, it changed only 0.016 percent of the weights, lifting its score on the MATH-500 benchmark. Astra also wrote small game-playing networks from scratch that outplayed networks trained for 100 million steps with PPO, a widely used reinforcement learning algorithm.
The findings matter because weight space has been a blind spot: even specialists cannot look at raw numbers and say what a network does. If language models can natively inspect and repair weights, they could become a new kind of optimizer and a practical tool for interpretability, the effort to understand how neural networks work inside. The authors stress the results are early and limited to small models, but they argue that deliberately reasoning about weights may soon complement, or even challenge, the gradient-based methods that have powered deep learning so far.