LLM Alignment Is Concentrated in the Final Layers

·2026.06.15 01:32

Key point

A new study finds that safety and sycophancy behaviors in LLMs are concentrated in the later layers of the model.

Details

The researcher proposes the 'Two-Body Hypothesis,' suggesting that a model's Capability production and Behavioral routing can be functionally separated.

Experimenting across several small-scale Transformer setups, the study found that Safety and Sycophancy-related properties peak near the end of the network (96–97% depth). Based on this, the following experiments were conducted:

Sparse intervention
Late-layer safety fine-tuning
Layer-frozen GRPO
Adapter merging

The core finding suggests that rather than a single 'alignment layer' existing in a model, spatially targeted alignment can be tested and may prove useful.

ai-safety mechanistic-interpretability llm-alignment transformer-architecture two-body-hypothesis

Read originalShare on Threads

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.