Recursive Scaling in Masked Diffusion Models

• URL: http://arxiv.org/abs/2606.18022v1
• Date: Tue, 16 Jun 2026 15:02:49 GMT
• Title: Recursive Scaling in Masked Diffusion Models
• Abstract summary: Masked diffusion models (MDMs) have recently emerged as a promising paradigm for sequence generation.<n>We introduce Recursive Masked Diffusion Models (R-MDMs) to increase model depth without increasing parameter count.<n>Across structured generation tasks, including Sudoku and Countdown, we show that R-MDMs achieve substantially improved parameter efficiency.
• Score: 67.93066069475753
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Masked diffusion models (MDMs) have recently emerged as a promising paradigm for sequence generation. Scaling MDMs is conventionally achieved by increasing the parameter count or the number of denoising steps. We introduce Recursive Masked Diffusion Models (R-MDMs), which add recursive depth as a third scaling axis by repeatedly applying the same denoising transformer within each diffusion step. Recursion enables iterative refinement of the output through parameter reuse, increasing effective model depth without increasing parameter count. Across structured generation tasks, including Sudoku and Countdown, we show that R-MDMs achieve substantially improved parameter efficiency: a model with $L$ recursive iterations often matches the performance of non-recursive baselines with roughly $L\times$ more parameters. Moreover, recursive refinement can partially substitute for additional denoising steps, allowing recursive models to reach the same generation quality with fewer forward passes at inference time. These results suggest that recursive depth is a practically useful scaling mechanism for MDMs, improving both parameter efficiency and the allocation of test-time compute.

Related papers

• Looped Diffusion Language Models [47.8566079047557]
  Masked diffusion models (MDMs) have emerged as a promising alternative to autoregressive models for language modeling.<n>We show that selectively looping the early-middle transformer layers significantly improves both training efficiency and model performance in MDMs.
  arXiv  Detail & Related papers  (2026-05-25T17:58:24Z)
• Generative Recursive Reasoning [67.22973831501257]
  Generative Recursive reAsoning Models (GRAM) is a framework that turns latent reasoning into probabilistic multi-trajectory.<n>GRAM supports conditional reasoning via $p_(y mid x)$ and, with fixed or absent inputs, unconditional generation via $p_(x)$.
  arXiv  Detail & Related papers  (2026-05-19T05:20:56Z)
• Recursive Multi-Agent Systems [106.82473907888497]
  We introduce RecursiveMAS, a multi-agent framework that casts the entire system as a unified latent-space computation.<n>RecursiveMAS connects heterogeneous agents as a collaboration loop through the lightweight RecursiveLink module.<n> Theoretical analyses of runtime complexity and learning dynamics establish that RecursiveMAS is more efficient than standard text-based MAS.
  arXiv  Detail & Related papers  (2026-04-28T17:59:34Z)
• Discrete Stochastic Localization for Non-autoregressive Generation [17.56505846228918]
  We show that emphtraining alone can substantially improve the step-efficiency of MDLM/ReMDM sampling.<n>On OpenWebText, textsc fine-tuning yields large MAUVE gains at low step budgets, surpassing the MDLM+ReMDM.<n>Analyses show improved self-correction and uncertainty calibration, making remasking markedly more compute-efficient.
  arXiv  Detail & Related papers  (2026-02-18T04:05:40Z)
• SpiralFormer: Looped Transformers Can Learn Hierarchical Dependencies via Multi-Resolution Recursion [24.26069897783496]
  SpiralFormer is a looped Transformer that executes recurrence under a multi-resolution recursion schedule.<n>We show that SpiralFormer achieves better parameter and compute efficiency than both looped and non-looped baselines across model scales from 160M to 1.4B.
  arXiv  Detail & Related papers  (2026-02-12T08:23:21Z)
• Looping Back to Move Forward: Recursive Transformers for Efficient and Flexible Large Multimodal Models [63.47909317137073]
  Large Multimodal Models (LMMs) have achieved remarkable success in vision-language computation tasks.<n>But their vast parameter counts are often underutilized during both training and inference.<n>We propose RecursiveVLM, a recursive Transformer architecture tailored for LMMs.
  arXiv  Detail & Related papers  (2026-02-09T17:58:23Z)
• MeSH: Memory-as-State-Highways for Recursive Transformers [23.995570647573484]
  Recursive models with fewer parameters often lag behind non-recursive counterparts under matched compute.<n>By probing hidden states, we trace this performance gap to two primary bottlenecks.<n>We introduce a Memory-as-State-Highways scheme, which externalizes state management into an explicit memory buffer.
  arXiv  Detail & Related papers  (2025-10-09T03:23:38Z)
• Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation [61.67090981767583]
  We introduce Mixture-of-Recursions (MoR), a unified framework that combines the two axes of efficiency inside a single Recursive Transformer.<n>MoR reuses a shared stack of layers across recursion steps to achieve parameter efficiency, while lightweight routers enable adaptive token-level thinking.<n>We also propose a KV sharing variant that reuses KV pairs from the first recursion, specifically designed to further decrease memory footprint.
  arXiv  Detail & Related papers  (2025-07-14T17:49:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.