Anthropic Open-Sources Jacobian Lens

Anthropic has released Jacobian Lens (J-lens) as a companion tool for language model interpretability research. The existing logit lens works by directly projecting intermediate activations onto the output vocabulary, assuming that all layers share the same coordinate system, which made interpretation difficult in early layers.

J-lens linearly transforms the residual stream vector at layer l into the final layer's basis using the average of the input-output Jacobian, then decodes it via unembedding. By averaging the Jacobian across many prompts and positions, it becomes possible to distinguish representations that are merely coincidentally activated from those that actually have the potential to be verbalized.

Key features:

• Visualizes top tokens per layer and position in a grid
• The lens can be trained with as few as about 100 prompts (the paper used 1,000)
• Generally applicable to HuggingFace decoder models (examples: Qwen family)
• Explicitly stated as a reference implementation, not accepting maintenance or contributions

This is companion code for the paper "Verbalizable Representations Form a Global Workspace in Language Models," released on GitHub.