Not All LLM Reasoning is Visible in the Chain-of-Thought

A cutting-edge study on AI security indicates that some large language models exhibit a failure mode of using semantically irrelevant filler words to perform invisible reasoning. Researchers evaluated the performance of 13 advanced models in three tasks and found that many models achieved significant performance improvements by adding specific filler words, with accuracy increases of up to 13 percentage points. This phenomenon depends on the tokens used and the specific model configuration; for example, filler words enabled Claude Opus 4.5 to satisfy hidden modular operation constraints without sacrificing accuracy in the main task, proving that invisible reasoning can serve goals that cannot be detected by CoT monitoring. Although reinforcement learning gave Qwen3-235B a strong preference for filler word content, neither reinforcement learning nor supervised fine-tuning resulted in persistent filler word benefits during testing. The results suggest that advanced models perform consequential calculations in their output tokens, but there are no explainable traces of reasoning.