EndoCoT: Scaling Endogenous Chain-of-Thought Reasoning in Diffusion Models

• URL: http://arxiv.org/abs/2603.12252v2
• Date: Mon, 16 Mar 2026 05:02:28 GMT
• Title: EndoCoT: Scaling Endogenous Chain-of-Thought Reasoning in Diffusion Models
• Abstract summary: We propose Endogenous Chain-of-Thought (EndoCoT) as a novel framework to activate MLLMs' reasoning potential.<n>EndoCoT bridges latent thought states through an iterative thought guidance module, and then bridges these states to the DiT's denoising process.<n>Extensive evaluations across diverse benchmarks achieve an average accuracy of 92.1%, outperforming the strongest baseline by 8.3 percentage points.
• Score: 40.37673945173621
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, Multimodal Large Language Models (MLLMs) have been widely integrated into diffusion frameworks primarily as text encoders to tackle complex tasks such as spatial reasoning. However, this paradigm suffers from two critical limitations: (i) MLLMs text encoder exhibits insufficient reasoning depth. Single-step encoding fails to activate the Chain-of-Thought process, which is essential for MLLMs to provide accurate guidance for complex tasks. (ii) The guidance remains invariant during the decoding process. Invariant guidance during decoding prevents DiT from progressively decomposing complex instructions into actionable denoising steps, even with correct MLLM encodings. To this end, we propose Endogenous Chain-of-Thought (EndoCoT), a novel framework that first activates MLLMs' reasoning potential by iteratively refining latent thought states through an iterative thought guidance module, and then bridges these states to the DiT's denoising process. Second, a terminal thought grounding module is applied to ensure the reasoning trajectory remains grounded in textual supervision by aligning the final state with ground-truth answers. With these two components, the MLLM text encoder delivers meticulously reasoned guidance, enabling the DiT to execute it progressively and ultimately solve complex tasks in a step-by-step manner. Extensive evaluations across diverse benchmarks (e.g., Maze, TSP, VSP, and Sudoku) achieve an average accuracy of 92.1%, outperforming the strongest baseline by 8.3 percentage points. The code and dataset are publicly available at https://lennoxdai.github.io/EndoCoT-Webpage/.

Related papers

• Escaping Confidence Trap: Evolutionary Decoding for Mathematical Reasoning in Diffusion LLMs [69.52853771994451]
  We analyze the decoding trajectories of LLaDA 2.0 and identify a recurring diffusion confidence trap.<n>We propose Evolutionary Decoding, a training-free test-time scaling framework that views diffusion decoding as an evolutionary process.
  arXiv  Detail & Related papers  (2026-08-01T11:48:25Z)
• Prism: Efficient Test-Time Scaling via Hierarchical Search and Self-Verification for Discrete Diffusion Language Models [96.0074341403456]
  Inference-time compute has re-emerged as a practical way to improve LLM reasoning.<n>Most test-time scaling (TTS) algorithms rely on autoregressive decoding.<n>We propose Prism, an efficient TTS framework for dLLMs.
  arXiv  Detail & Related papers  (2026-02-02T09:14:51Z)
• Residual Context Diffusion Language Models [90.07635240595926]
  Residual Context Diffusion (RCD) is a module that converts discarded token representations into contextual residuals and injects them back for the next denoising step.<n>RCD consistently improves frontier dLLMs by 5-10 points in accuracy with minimal extra computation overhead.
  arXiv  Detail & Related papers  (2026-01-30T13:16:32Z)
• TIT: A Tree-Structured Instruction Tuning Approach for LLM-Based Code Translation [11.882496324328905]
  We propose TIT, a Tree-structured Instruction Tuning paradigm for LLM-based code translation.<n>To mitigate syntactic confusion, the syntactic information representation module integrates language-agnostic syntactic features.<n>To generate high-quality fine-grained parallel data, the fine-grained parallel dataset augmentation module aligns nodes with code segments.
  arXiv  Detail & Related papers  (2025-10-10T13:53:46Z)
• Diffusion Language Models Know the Answer Before Decoding [56.96815863705218]
  Diffusion language models (DLMs) have emerged as an alternative to autoregressive approaches.<n>Our work highlights and leverage an overlooked property of DLMs early answer convergence.<n>We introduce Prophet, a training-free fast decoding paradigm that enables early commit decoding.
  arXiv  Detail & Related papers  (2025-08-27T15:40:25Z)
• LLM as Effective Streaming Processor: Bridging Streaming-Batch Mismatches with Group Position Encoding [29.586274567275012]
  It is commonly assumed that the latter two mismatches require frequent re-encoding, indicating re-encoding outputs is largely unnecessary.<n>We introduce a group position encoding paradigm built on batch architectures to enhance consistency between streaming and batch modes.<n>Our method requires no architectural modifications, exhibits strong generalization in both streaming and batch modes.
  arXiv  Detail & Related papers  (2025-05-22T17:53:28Z)
• SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMs [48.28847964704554]
  Chain-of-Thought (CoT) reasoning enables Large Language Models (LLMs) to solve complex reasoning tasks.<n>We propose a novel approach for continuous-space reasoning that does not require modifying the LLM.
  arXiv  Detail & Related papers  (2025-02-17T18:52:29Z)
• Gap-Filling Prompting Enhances Code-Assisted Mathematical Reasoning [0.0]
  Chain-of-thought (CoT) and program-of-thought (PoT) fine-tuning are common methods to transfer LLM knowledge to small language models (SLMs) This paper introduces Gap-Filling Prompting (GFP), a novel two-step prompting strategy designed to enhance the problem-solving process for SLMs.
  arXiv  Detail & Related papers  (2024-11-08T08:52:59Z)
• Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding [46.485363806259265]
  Speculative Decoding has emerged as a novel decoding paradigm for Large Language Models (LLMs) inference. In each decoding step, this method first drafts several future tokens efficiently and then verifies them in parallel. This paper presents a comprehensive overview and analysis of this promising decoding paradigm.
  arXiv  Detail & Related papers  (2024-01-15T17:26:50Z)
• Guiding the PLMs with Semantic Anchors as Intermediate Supervision: Towards Interpretable Semantic Parsing [57.11806632758607]
  We propose to incorporate the current pretrained language models with a hierarchical decoder network. By taking the first-principle structures as the semantic anchors, we propose two novel intermediate supervision tasks. We conduct intensive experiments on several semantic parsing benchmarks and demonstrate that our approach can consistently outperform the baselines.
  arXiv  Detail & Related papers  (2022-10-04T07:27:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.