AI BriefingLinear Program Optimizes GPT-6 Astra's Weight Edit with 8x Fewer Changed Weights
·2026.11.09 09:00
Key point
A linear program optimizes the weight edit found by GPT-6 Astra on Qwen3.5-0.8B, achieving comparable math performance with 8x fewer changed weights but slightly lower instruction-following scores.
Details
- A linear program (LP) replaces the trial-and-error search used by GPT-6 Astra to determine weight changes, achieving comparable results on math benchmarks with significantly fewer changes.
- The LP edit changes only 16,291 weights in Qwen3.5-0.8B, compared to 130,188 for Astra's edit. It matches or exceeds Astra's accuracy on GSM8K (69.6 vs 68.2) and MATH-500 (51.7 vs 50.9), but scores lower on IFEval (59.7 vs 61.9).
- The method relies on identifying a loop direction vector and specific layers; the best results use Astra's direction and layers. Using a heuristic direction instead of Astra's reduces performance to about 93% of the full fix.
- A simple steering vector added at inference time recovers most of the benefit without modifying the checkpoint, though the LP edit itself is an ordinary checkpoint.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
OpenAI Disbands Robotics Team, Pivots to General Models for Embodied AINew Arm NEON kernel speeds up llama.cpp prompt processing by up to 6.5x