The Anatomy of the Refusal Circuit

Tracking the refusal mechanism across 12 models, 6 labs, ranging from 2B to 72B, refusal behavior was repeatedly observed as a sparse gate-amplifier circuit.

• A mid-layer gate attention head reads the detection-layer representation and produces a routing vector.
• The subsequent amplifier heads amplify that signal, leading to refusal / censorship behavior.

What stands out is also clear.

• Looking at output DLA alone, the gate is almost invisible. In Qwen3-8B, the gate's contribution is under 1%, so it doesn't appear as a top attention head.
• However, interchange testing confirms the gate is causal. Turning off this head weakens the downstream amplifiers.
• As scale increases, per-head ablation rapidly weakens. Among the tested scaling pairs, the difference was as much as 58x, and at 72B, the top per-head ablation appears almost like noise.
• In contrast, interchange still identifies the trigger component.
• When a bijection encoding is given in-context and the prompt is also encoded that way, the need for the gate collapses, and the model shifts from refusal to puzzle-solving instead.

The interpretation the authors propose is that detection and policy routing are separate computations, and the refusal routing circuit becomes fixed early on. If the input fails to produce a representation the gate can read at that point, the downstream policy fails to properly engage.