State of Thought Enables Endogenous Reasoning
Abstract Overview
This paper introduces State of Thought (SoT), a test-time reasoning framework in which an LLM's internal reasoning state governs which prior evidence remains active and when reasoning should stop. Rather than relying on externally scripted reasoning formats or broad search over many trajectories, SoT reads a compact four-dimensional dynamics-geometric state and uses a lightweight 582-parameter controller on frozen backbone models. The method is evaluated on 3 LLMs across 16 datasets spanning quantitative, general-understanding, symbolic/code, and long-context reasoning, and is also transferred to 2 vision-language model scales over 3 reasoning tasks. The reported evidence frames reasoning as state-conditioned evidence organization, aiming to improve both accuracy and inference efficiency.
Novelty
The distinctive contribution is the shift from externally imposed reasoning programs or search policies to endogenous, state-conditioned control of evidence selection and stopping. The approach implements this via a compact 582-parameter controller on frozen backbones, preserving interpretability by explicitly activating textual evidence rather than routing reasoning through opaque latent chains.
Results
Across the 16-dataset LLM evaluation, SoT consistently improves mean-baseline accuracy while reducing generated tokens by 62.6% and end-to-end latency by 44.6%. On Llama-3.1-8B, it achieves the best domain-level average in all four reasoning domains and the best or tied-best result on 13 of 16 datasets. In vision-language transfer across two model scales and three tasks, it averages 87.5% accuracy and achieves 74.9% fewer completion tokens with 73.5% lower latency compared to search-based baselines.
Key Points
- SoT uses a compact four-dimensional endogenous reasoning state to govern sparse evidence activation and stopping rather than relying on external scripts or large search budgets.
- The method keeps the underlying backbone frozen and adds a 582-parameter controller, with ablations confirming that evidence organization and stopping address distinct failure modes.
- Empirical evaluations across LLMs and VLMs demonstrate simultaneous gains in accuracy and efficiency, while extensions show viability in training-free, embedding-only, and black-box trajectory judging setups.