Generative Recursive Reasoning

• URL: http://arxiv.org/abs/2605.19376v2
• Date: Wed, 20 May 2026 08:03:28 GMT
• Title: Generative Recursive Reasoning
• Abstract summary: Generative Recursive reAsoning Models (GRAM) is a framework that turns latent reasoning into probabilistic multi-trajectory.<n>GRAM supports conditional reasoning via $p_(y mid x)$ and, with fixed or absent inputs, unconditional generation via $p_(x)$.
• Score: 67.22973831501257
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: How should future neural reasoning systems implement extended computation? Recursive Reasoning Models (RRMs) offer a promising alternative to autoregressive sequence extension by performing iterative latent-state refinement with shared transition functions. Yet existing RRMs are largely deterministic, following a single latent trajectory and converging to a single prediction. We introduce Generative Recursive reAsoning Models (GRAM), a framework that turns recursive latent reasoning into probabilistic multi-trajectory computation. GRAM models reasoning as a stochastic latent trajectory, enabling multiple hypotheses, alternative solution strategies, and inference-time scaling through both recursive depth and parallel trajectory sampling. This yields a latent-variable generative model supporting conditional reasoning via $p_θ(y \mid x)$ and, with fixed or absent inputs, unconditional generation via $p_θ(x)$. Trained with amortized variational inference, GRAM improves over deterministic recurrent and recursive baselines on structured reasoning and multi-solution constraint satisfaction tasks, while demonstrating an unconditional generation capability. https://ahn-ml.github.io/gram-website

Related papers

• Think Deep, Speak Once: Relit, A Recursive Latent Implicit Transformer Framework [2.007262412327553]
  We introduce ReLIT, a hybrid framework for deep, semantically grounded implicit reasoning.<n>We show that ReLIT can be scaled efficiently through recurrent depth rather than parameter width.<n>These results demonstrate that reasoning capability can be scaled efficiently through recurrent depth rather than parameter width.
  arXiv  Detail & Related papers  (2026-08-08T12:50:51Z)
• Recursive Vision Language Models for General Symbolic Reasoning [17.777375944237015]
  Symbolic-reasoning tasks such as Sudoku, maze pathfinding, and ARC remain challenging for LLMs.<n>We propose R-Qwen, a framework built upon a pretrained Qwen backbone.<n>R-Qwen repeatedly refines a candidate solution through programmatic self-recursion and deep supervision.
  arXiv  Detail & Related papers  (2026-08-02T23:01:10Z)
• HoloRec: Holistic Encoding and Interleaved Reasoning for Generative Recommendation [57.802158017741824]
  HoloRec is an endogenous chain-of-thought recommendation mechanism that unifies representation, reasoning, and generation.<n> Experiments on multiple public recommendation datasets demonstrate that HoloRec consistently outperforms baselines.
  arXiv  Detail & Related papers  (2026-06-13T14:52:03Z)
• Recursive Multi-Agent Systems [106.82473907888497]
  We introduce RecursiveMAS, a multi-agent framework that casts the entire system as a unified latent-space computation.<n>RecursiveMAS connects heterogeneous agents as a collaboration loop through the lightweight RecursiveLink module.<n> Theoretical analyses of runtime complexity and learning dynamics establish that RecursiveMAS is more efficient than standard text-based MAS.
  arXiv  Detail & Related papers  (2026-04-28T17:59:34Z)
• Recursive Models for Long-Horizon Reasoning [28.82044197167549]
  We show that a model can invoke itself to solve subtasks in isolated contexts.<n>We generalize our framework to modern agentic systems with arbitrary context processing and control flows.
  arXiv  Detail & Related papers  (2026-03-02T17:37:10Z)
• Parallel Test-Time Scaling for Latent Reasoning Models [58.428340345068214]
  Parallel test-time scaling (TTS) is a pivotal approach for enhancing large language models (LLMs)<n>Recent advances in latent reasoning, where intermediate reasoning unfolds in continuous vector spaces, offer a more efficient alternative to explicit Chain-of-Thought.<n>This work enables parallel TTS for latent reasoning models by addressing the above issues.
  arXiv  Detail & Related papers  (2025-10-09T03:33:00Z)
• Statistical-Computational Trade-offs for Recursive Adaptive Partitioning Estimators [11.77169131123735]
  We show that greedy algorithms for high-dimensional regression get stuck at local optima.<n>We show that greedy training requires $exp(Omega(d))$ to achieve low estimation error.<n>This dichotomy mirrors that of two-layer neural networks trained with gradient descent (SGD) in the mean-field regime.
  arXiv  Detail & Related papers  (2024-11-07T03:11:53Z)
• COrAL: Order-Agnostic Language Modeling for Efficient Iterative Refinement [80.18490952057125]
  Iterative refinement has emerged as an effective paradigm for enhancing the capabilities of large language models (LLMs) on complex tasks. We propose Context-Wise Order-Agnostic Language Modeling (COrAL) to overcome these challenges. Our approach models multiple token dependencies within manageable context windows, enabling the model to perform iterative refinement internally.
  arXiv  Detail & Related papers  (2024-10-12T23:56:19Z)
• Amortizing intractable inference in large language models [56.92471123778389]
  We use amortized Bayesian inference to sample from intractable posterior distributions. We empirically demonstrate that this distribution-matching paradigm of LLM fine-tuning can serve as an effective alternative to maximum-likelihood training. As an important application, we interpret chain-of-thought reasoning as a latent variable modeling problem.
  arXiv  Detail & Related papers  (2023-10-06T16:36:08Z)
• Recursive Reinforcement Learning [4.429642479975602]
  Recursion is the fundamental paradigm to finitely describe potentially infinite objects. We develop RL algorithms capable of computing optimal policies in environments described as a collection of Markov decision processes.
  arXiv  Detail & Related papers  (2022-06-23T00:29:42Z)
• TreeFlow: Going beyond Tree-based Gaussian Probabilistic Regression [0.0]
  We introduce TreeFlow, the tree-based approach that combines the benefits of using tree ensembles with the capabilities of modeling flexible probability distributions. We evaluate the proposed method on challenging regression benchmarks with varying volume, feature characteristics, and target dimensionality.
  arXiv  Detail & Related papers  (2022-06-08T20:06:23Z)
• Scaling Structured Inference with Randomization [64.18063627155128]
  We propose a family of dynamic programming (RDP) randomized for scaling structured models to tens of thousands of latent states. Our method is widely applicable to classical DP-based inference. It is also compatible with automatic differentiation so can be integrated with neural networks seamlessly.
  arXiv  Detail & Related papers  (2021-12-07T11:26:41Z)
• A Variational Inference Approach to Inverse Problems with Gamma Hyperpriors [60.489902135153415]
  This paper introduces a variational iterative alternating scheme for hierarchical inverse problems with gamma hyperpriors. The proposed variational inference approach yields accurate reconstruction, provides meaningful uncertainty quantification, and is easy to implement.
  arXiv  Detail & Related papers  (2021-11-26T06:33:29Z)
• Probabilistic Circuits for Variational Inference in Discrete Graphical Models [101.28528515775842]
  Inference in discrete graphical models with variational methods is difficult. Many sampling-based methods have been proposed for estimating Evidence Lower Bound (ELBO) We propose a new approach that leverages the tractability of probabilistic circuit models, such as Sum Product Networks (SPN) We show that selective-SPNs are suitable as an expressive variational distribution, and prove that when the log-density of the target model is aweighted the corresponding ELBO can be computed analytically.
  arXiv  Detail & Related papers  (2020-10-22T05:04:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.