MIT CSAIL Research Demonstrates LLMs Can Natively Read and Write Neural Network Weights

MIT CSAIL researchers introduced Weight-Space Programming, a paradigm where frontier LLMs like GPT-6 Astra directly interpret and modify neural network weights without environment feedback or gradients.
Astra successfully identified the training purpose of various networks, including game-playing MLPs and language models, by analyzing weight distributions and attention patterns.
In a key optimization test, Astra improved Qwen3.5-0.8B's mathematical reasoning by removing looping patterns in specific MLP layers, boosting MATH-500 accuracy from 7.9% to 48.1% and IFEval scores by threefold.
The model demonstrated zero-shot weight writing, creating game-playing networks from scratch that outperformed PPO-trained baselines (trained for 100 million steps) in Breakout, Seaquest, and Asterix.

Technical Capabilities

Reading weights: Astra identified the training language of character-level transformers and located specific induction heads without executing the network or accessing input/output layers.
Writing weights: Astra repaired a failed physics control network (Kinetix) by editing its weights to achieve a 100% success rate, and generated Go-playing networks using a Python sandbox.
Tool usage: For large-scale tasks, Astra utilized a Python sandbox to read, analyze, and write weights, with performance improving as the allowed forward pass budget increased.

The technique is effective for removing existing obstacles or steering capabilities but cannot add fundamental abilities the model lacks, as shown by failed improvements on a non-looping Gemma 3 270M model.
Astra failed to identify sycophancy in fine-tuned models using weights alone, requiring access to attention patterns on specific text to distinguish between honest and sycophantic versions.
All experiment logs and code are publicly available on GitHub for reproducibility.