AI BriefingTree-Ising Repairs LLM Weights via Discrete Optimization, Outperforming Agent-Based Edits on MATH-500
·2026.11.11 22:32
Key point
The method uses simulated annealing on BF16 weights to fix reasoning loops in Qwen3.5-0.8B, requiring only $0.90 in GPU time and zero LLM tokens.
Details
- Tree-Ising casts LLM weight repair as a sparse discrete optimization problem over BF16 weights, decomposing it into independent QUBO problems solved via simulated annealing or quantum-inspired simulated bifurcation.
- On Qwen3.5-0.8B, the method achieves 54.6% on MATH-500, outperforming the GPT-6 Astra agent's 51.7% and the base model's 7.9%, while modifying 13% to 17% fewer weights and reducing bit flips by a factor of 2.2 to 2.3.
- The approach requires only four forward-pass requests and zero LLM tokens, with the annealing recipe costing approximately $0.90 in GPU time.
- On Qwen3.5-2B, the same recipe increases greedy GSM8K accuracy from 21.6% to 69.4% when layers and dose are selected using only the base model.
- An eight-variable repair subproblem was executed using QAOA on an IBM Heron processor as a proof of concept.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
MIT CSAIL Research Demonstrates LLMs Can Natively Read and Write Neural Network WeightsLinear Program Optimizes GPT-6 Astra's Weight Edit with 8x Fewer Changed Weights