Chain-of-Thought Reasoning in Real-World Environments Is Not Always Faithful

Recent research indicates that LLM Chain-of-Thought (CoT) outputs may not accurately reflect the actual process by which the model reaches a conclusion. While previous studies pointed out unfaithfulness in artificial biases or adversarial prompts, this study demonstrates that such phenomena also occur in non-adversarial prompts within natural contexts.

The research team discovered the phenomenon of Implicit Post-Hoc Rationalization, where models generate superficially consistent logic while producing contradictory conclusions (both Yes or both No) when presented with opposing questions such as "Is X greater than Y?" and "Is Y greater than X?". This is analyzed as being influenced by the model's inherent bias toward Yes or No.

Key findings are as follows:

• Rate of Unfaithful CoT Occurrence: Observed up to 13% in production models.
• Frontier Model Status: Even the latest models, such as DeepSeek R1 (0.37%) and Sonnet 3.7 with thinking (0.04%), do not guarantee complete faithfulness.
• Illogical Shortcuts: Subtle logical errors were also confirmed in difficult math problems, where speculative answers are made to appear rigorously proven.

These results suggest that while CoT may be useful for output evaluation, it does not fully explain the model's internal process, and thus should be used with caution in agent-based or safety-critical environments.