New Paper: Open 27B Model Achieves Self-Rewriting Without Gradients

·2026.11.21 01:29

Key point

An open 27B model improves its own accuracy by 6.6 points on unseen problems by rewriting its weights using only forward passes.

Details

  • Researchers introduce self-rewriting, a method where a language model edits its own weights to improve performance without gradients or training runs.
  • An open 27B model increased its accuracy by 6.6 points on unseen problems after applying its own weight edits.
  • The model repaired a damaged copy of itself, restoring over 80% of original accuracy on image tasks without retraining or accessing original weights.
  • Self-rewriting ability emerges only in the largest models of a family; smaller models fail to improve or break themselves during the process.
  • Unlike test-time scaling, the gains are written into the weights at no per-answer cost, and unlike training, the process uses only forward passes similar to standard serving.

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.