Alignment Only Increased Decisiveness

Post-training made LLM outputs more decisive, but it did not improve accuracy.

Comparing 3 architectures and 4 RL methods, the commitment layer where the model finally locks in its prediction did not shift position even after reinforcement learning.

What changed was the representational structure at that point.

• At that layer, the representation was monotonically compressed.
• The earlier layers that choose what to say remained largely unchanged.

In conclusion, reinforcement learning increased the model's decisiveness but did not raise its accuracy.