DreamReasoner-8B: Block-Size Curriculum Learning for Diffusion Reasoning Models

• URL: http://arxiv.org/abs/2606.19257v1
• Date: Wed, 17 Jun 2026 16:34:02 GMT
• Title: DreamReasoner-8B: Block-Size Curriculum Learning for Diffusion Reasoning Models
• Abstract summary: We develop DreamReasoner-8B, an open-source block diffusion reasoning model.<n>Our analysis reveals a stark performance disparity: training with large block sizes yields remarkably poor reasoning.<n>We propose block-size curriculum learning, which gradually transitions training from fine-grained to coarse-grained block sizes.
• Score: 82.86749180125919
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Block diffusion language models accelerate decoding through parallel block-wise denoising, yet whether they can be reliably scaled for long chain-of-thought (CoT) reasoning remains unresolved. To this end, we develop DreamReasoner-8B, an open-source block diffusion reasoning model, and conduct a systematic study of how training and inference block sizes affect long-CoT reasoning. Our analysis reveals a stark performance disparity: training with large block sizes yields remarkably poor reasoning, whereas small block sizes preserve effective reasoning. To bridge this granularity gap, we propose block-size curriculum learning, which gradually transitions training from fine-grained to coarse-grained block sizes, thereby overcoming this limitation and enabling strong reasoning performance that generalizes across diverse inference block sizes. On mathematical and code reasoning benchmarks, DreamReasoner-8B achieves results competitive with leading open autoregressive models such as Qwen3-8B. This work establishes a practical foundation for efficient, reasoning-capable diffusion language models. We release our model at https://github.com/DreamLM/DreamReasoner.

Related papers

• Break the Block: Dynamic-size Reasoning Blocks for Diffusion Large Language Models via Monotonic Entropy Descent with Reinforcement Learning [41.859993506122194]
  The fixed-size block generations remain a critical bottleneck for effective and coherent reasoning.<n>This paper proposes b1, a novel post-training framework for dLLMs that learns dynamic-size reasoning blocks via a Monotonic Entropy Descent objective.
  arXiv  Detail & Related papers  (2026-05-04T06:17:49Z)
• Advancing Block Diffusion Language Models for Test-Time Scaling [73.54022593833638]
  We propose a unified framework for test-time scaling in BDLMs.<n>We introduce adaptivity in both decoding and block-wise generation.<n>We show that applying BACD and TCCF to TDAR-8B yields significant improvements over strong baselines.
  arXiv  Detail & Related papers  (2026-02-10T09:05:07Z)
• FlashBlock: Attention Caching for Efficient Long-Context Block Diffusion [51.1618564189244]
  FlashBlock is a cached block-external attention mechanism that reuses stable attention output, reducing attention computation and KV cache access without modifying the diffusion process.<n> Experiments on diffusion language models and video generation demonstrate up to 1.44$times$ higher token throughput and up to 1.6$times$ reduction in attention time, with negligible impact on generation quality.
  arXiv  Detail & Related papers  (2026-02-05T04:57:21Z)
• Blockwise SFT for Diffusion Language Models: Reconciling Bidirectional Attention and Autoregressive Decoding [60.06816407728172]
  Discrete diffusion language models have shown strong potential for text generation.<n>Standard supervised fine-tuning misaligns with semi-autoregressive inference.<n>We propose Blockwise SFT, which partitions responses into fixed-size blocks.
  arXiv  Detail & Related papers  (2025-08-27T02:49:33Z)
• Think in Blocks: Adaptive Reasoning from Direct Response to Deep Reasoning [3.773711855945839]
  Large Language Models (LLMs) with chains-of-thought have demonstrated strong performance on an increasing range of tasks.<n>This raises a question: can LLMs dynamically adjust the length of their reasoning processes based on task complexity?<n>We propose the Think in Blocks framework, which enables adaptive reasoning-from zero to deep reasoning-by partitioning the reasoning process into a tunable number of blocks.
  arXiv  Detail & Related papers  (2025-08-21T12:32:19Z)
• DiffusionBlocks: Block-wise Neural Network Training via Diffusion Interpretation [11.910667302899638]
  DiffusionBlocks is a principled framework for transforming transformer-based networks into genuinely independent trainable blocks.<n>Our experiments on a range of transformer architectures demonstrate that DiffusionBlocks training matches the performance of end-to-end training.
  arXiv  Detail & Related papers  (2025-06-17T05:44:18Z)
• Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models [15.853201399662344]
  Diffusion language models offer unique benefits over autoregressive models.<n>They lag in likelihood modeling and are limited to fixed-length generation.<n>We introduce a class of block diffusion language models that interpolate between discrete denoising diffusion and autoregressive models.
  arXiv  Detail & Related papers  (2025-03-12T17:43:40Z)
• Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language Models [100.53662473219806]
  Diffusion-of-Thought (DoT) is a novel approach that integrates diffusion models with Chain-of-Thought.<n>DoT allows reasoning steps to diffuse over time through a diffusion language model.<n>Our results demonstrate the effectiveness of DoT in multi-digit multiplication, logic, and grade school math problems.
  arXiv  Detail & Related papers  (2024-02-12T16:23:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.