One Step Forward and K Steps Back: Better Reasoning with Denoising Recursion Models

• URL: http://arxiv.org/abs/2604.18839v1
• Date: Mon, 20 Apr 2026 21:06:12 GMT
• Title: One Step Forward and K Steps Back: Better Reasoning with Denoising Recursion Models
• Abstract summary: We introduce Denoising Recursion Models, a method that corrupts data with noise but trains the model to reverse the corruption over multiple steps.<n>This strategy provides a tractable curriculum of intermediate states, while better aligning training with testing and incentivizing non-greedy, forward-looking generation.<n>We show this approach outperforms the Tiny Recursion Model on ARC-AGI, where it recently achieved breakthrough performance.
• Score: 4.903188186588148
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Looped transformers scale computational depth without increasing parameter count by repeatedly applying a shared transformer block and can be used for iterative refinement, where each loop rewrites a full fixed-size prediction in parallel. On difficult problems, such as those that require search-like computation, reaching a highly structured solution starting from noise can require long refinement trajectories. Learning such trajectories is challenging when training specifies only the target solution and provides no supervision over the intermediate refinement path. Diffusion models tackle this issue by corrupting data with varying magnitudes of noise and training the model to reverse it in a \textit{single step}. However, this process misaligns training and testing behaviour. We introduce Denoising Recursion Models, a method that similarly corrupts data with noise but trains the model to reverse the corruption over \textit{multiple} recursive steps. This strategy provides a tractable curriculum of intermediate states, while better aligning training with testing and incentivizing non-greedy, forward-looking generation. Through extensive experiments, we show this approach outperforms the Tiny Recursion Model (TRM) on ARC-AGI, where it recently achieved breakthrough performance.

Related papers

• Focus When Necessary: Adaptive Routing and Collaborative Grounding for Training-Free Visual Grounding [56.38477453373368]
  LazyMCoT is a dynamic and training-free framework that adaptively allocates visual grounding efforts based on sample difficulty.<n>LazyMCoT rivals training-based approaches by simultaneously improving reasoning accuracy and reducing average inference latency.
  arXiv  Detail & Related papers  (2026-06-15T03:17:44Z)
• Exploiting the Exact Denoising Posterior Score in Training-Free Guidance of Diffusion Models [0.0]
  A popular class of methods, based on Diffusion Posterior Sampling (DPS), attempts to approximate the intractable posterior score function directly.<n>We present a novel expression for the exact posterior score for purely denoising tasks that is tractable in terms of the unconditional score function.<n>We demonstrate that these step sizes are transferable to related inverse problems such as colorization, random inpainting, and super resolution.
  arXiv  Detail & Related papers  (2025-06-16T15:43:28Z)
• Test-Time Scaling of Diffusion Models via Noise Trajectory Search [10.8507840358202]
  We introduce an $epsilon$-greedy search algorithm that globally explores at extreme timesteps and locally exploits during the intermediate steps where de-mixing occurs.<n>Experiments on EDM and Stable Diffusion reveal state-of-the-art scores for class-conditioned/text-to-image generation.
  arXiv  Detail & Related papers  (2025-05-24T19:13:29Z)
• Half-order Fine-Tuning for Diffusion Model: A Recursive Likelihood Ratio Optimizer [16.103949557802988]
  probabilistic diffusion model (DM) generates content by inferencing through a chain structure.<n>Modern methods are either based on Reinforcement Learning (RL) or truncated Backpropagation (BP)<n>We propose the Recursive Likelihood Ratio (RLR) fine-tuning paradigm for DM.
  arXiv  Detail & Related papers  (2025-02-02T03:00:26Z)
• Truncated Consistency Models [57.50243901368328]
  Training consistency models requires learning to map all intermediate points along PF ODE trajectories to their corresponding endpoints.<n>We empirically find that this training paradigm limits the one-step generation performance of consistency models.<n>We propose a new parameterization of the consistency function and a two-stage training procedure that prevents the truncated-time training from collapsing to a trivial solution.
  arXiv  Detail & Related papers  (2024-10-18T22:38:08Z)
• Improving Diffusion Inverse Problem Solving with Decoupled Noise Annealing [84.97865583302244]
  We propose a new method called Decoupled Annealing Posterior Sampling (DAPS)<n>DAPS relies on a novel noise annealing process.<n>We demonstrate that DAPS significantly improves sample quality and stability across multiple image restoration tasks.
  arXiv  Detail & Related papers  (2024-07-01T17:59:23Z)
• An Efficient Rehearsal Scheme for Catastrophic Forgetting Mitigation during Multi-stage Fine-tuning [55.467047686093025]
  A common approach to alleviate such forgetting is to rehearse samples from prior tasks during fine-tuning.<n>We propose a sampling scheme, textttbf mix-cd, that prioritizes rehearsal of collateral damage'' samples.<n>Our approach is computationally efficient, easy to implement, and outperforms several leading continual learning methods in compute-constrained settings.
  arXiv  Detail & Related papers  (2024-02-12T22:32:12Z)
• Noisy Pair Corrector for Dense Retrieval [59.312376423104055]
  We propose a novel approach called Noisy Pair Corrector (NPC) NPC consists of a detection module and a correction module. We conduct experiments on text-retrieval benchmarks Natural Question and TriviaQA, code-search benchmarks StaQC and SO-DS.
  arXiv  Detail & Related papers  (2023-11-07T08:27:14Z)
• Dynamic Residual Classifier for Class Incremental Learning [4.02487511510606]
  With imbalanced sample numbers between old and new classes, the learning can be biased. Existing CIL methods exploit the longtailed (LT) recognition techniques, e.g., the adjusted losses and the data re-sampling methods. A novel Dynamic Residual adaptation (DRC) is proposed to handle this challenging scenario.
  arXiv  Detail & Related papers  (2023-08-25T11:07:11Z)
• Latent Class-Conditional Noise Model [54.56899309997246]
  We introduce a Latent Class-Conditional Noise model (LCCN) to parameterize the noise transition under a Bayesian framework. We then deduce a dynamic label regression method for LCCN, whose Gibbs sampler allows us efficiently infer the latent true labels. Our approach safeguards the stable update of the noise transition, which avoids previous arbitrarily tuning from a mini-batch of samples.
  arXiv  Detail & Related papers  (2023-02-19T15:24:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.