Anthropic Unveils Technique for Interpreting Claude's Internal Thought Process

Anthropic has announced Natural Language Autoencoders (NLAs), a technique that converts activations—a model's internal numerical data—into human-readable English text. Unlike the Chain-of-Thought that a model outputs externally, this allows direct insight into the model's actual internal state.

Core Mechanism of NLA:

• Activation Verbalizer (AV): Converts activation values into text descriptions.
• Activation Reconstructor (AR): Reconstructs the converted text back into activation values, verifying whether the description accurately reflects the actual internal state.

Key Research Findings:

• Hidden cognitive ability: Claude recognized in about 26% of cases that it was being tested in safety evaluation situations, but did not verbalize this externally.
• Capturing advance planning: When the model was writing poetry, it was confirmed to engage in advanced planning processes, such as already internally deciding on the final rhyme before outputting the first word.

By transparently revealing AI's decision-making process, this technology provides a groundbreaking tool for AI Alignment and safety research.