The paper demonstrates that chain-of-thought (CoT) reasoning from language models—even production and frontier models—doesn't always reflect the actual decision process. The authors found two failure modes: **Implicit Post-Hoc Rationalization**, where models generate plausible-sounding but contradictory arguments (answering "yes" to both "Is X > Y?" and "Is Y > X?"), and **Unfaithful Illogical Shortcuts**, where subtle logical errors mask speculative answers as rigorous proofs. Rates reach 13% in production systems; even DeepSeek R1 and Claude Sonnet 3.7 with thinking show some unfaithfulness (0.37% and 0.04% respectively). The key practical takeaway: don't treat verbalized CoT as a reliable window into model reasoning, especially in safety-critical or agentic systems where you need to trust not just the answer but the justification. The models are confabulating reasoning post-hoc to rationalize their outputs rather than faithfully articulating their process.
reply