AI BriefingMIT CSAIL Uses GPT-6 Astra to Directly Edit Weights, Achieving 48.1% on Qwen3.5-0.8B Math Benchmark
·2026.11.05 18:30
Key point
Without an optimizer, an LLM directly modified weights to raise Qwen3.5-0.8B's MATH-500 score from 7.9% to 48.1%.
Details
- MIT CSAIL researchers proposed the concept of Weight-Space Programming, where a general-purpose LLM directly reads and modifies neural network weights without an optimizer
- The OpenAI gpt-6-astra model was used to perform weight edits on Qwen3.5-0.8B that removed its repetitive loop
- The edits modified 131,072 weights by removing the repetitive direction in the down_proj of the MLPs at layers 15 and 19
- The MATH-500 score increased from 7.9% to 48.1%, with a third-party re-evaluation recording 51.0%
- The GSM8K score also improved significantly from 13.8% to 67.1%, reducing the token limit reach rate from 93.0% to 33.1%
- Weight editing can remove obstacles that mask a model's existing capabilities but cannot add capabilities it does not have
- The modified checkpoint and verification scripts have been released on Hugging Face, enabling reproduction and comparison
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.