Qwen Releases SAE Interpretability Module for Qwen3.5-27B

Qwen has released Qwen-Scope. It inserts a Sparse Autoencoder (SAE) into the hidden layers of the Qwen3 and Qwen3.5 model families to interpret internal representations, enabling their use for control, analysis, and optimization.

• Target base model: Qwen3.5-27B
• SAE width: 81,920
• hidden size: 5,120
• expansion factor: 16x
• Top-K: 100
• hook point: residual stream
• Applied layers: 0~63

The checkpoint for each layer is a PyTorch dict in the format layer{n}.sae.pt, containing 4 tensors: W_enc, W_dec, b_enc, b_dec.

The documentation also provides an example of hooking the residual stream to extract feature activations, along with instructions for running the Gradio demo via app.py.