The Impact of LLM 'Linguistic Illegibility' on Security: Limitations of CoT Monitoring and Proposed Isolation Techniques

The paper 'The Implications of Linguistic Illegibility for LLM Security', submitted to arXiv by James Mickens, analyzes fundamental limitations in LLM security and proposes new defense strategies.

The Concept of Linguistic Illegibility

The paper introduces the concept of linguistic illegibility. This refers to the phenomenon where externalized language outputs or mechanically extracted linguistic features of an LLM fail to reliably represent the model's actual internal computation methods.

• Cause: LLM internal computations are performed as mathematical operations in the activation space, not in natural language.
• Loss: There is a lossy translation process between the activation space and natural language output.

Limitations of Existing Security Mechanisms

Due to these characteristics, security mechanisms relying on the model's linguistic self-reporting cannot be fully sound. The paper points out that the following methods are unreliable due to the influence of linguistic illegibility:

• Chain-of-thought (CoT) monitoring
• Constitutional self-critique
• Activation probing on linguistically-defined feature vectors

Proposed Alternative: Isolation Techniques

The paper proposes isolation techniques as an alternative that provides guarantees without reading the model's linguistic state.

• Taint Tracking: Applying policies that pre-define pieces of system state that should not be influenced by model-generated data, regardless of how the model linguistically self-reports.
• Additional Defenses: Robust virtualization, third-party auditing of sandbox configurations, etc.

These mechanisms are evaluated as providing a critical floor even under language-based monitoring, and may have defended against sandbox exploits recently discovered in frontier models.