Recursive Vision Language Models for General Symbolic Reasoning

• URL: http://arxiv.org/abs/2608.01534v1
• Date: Sun, 02 Aug 2026 23:01:10 GMT
• Title: Recursive Vision Language Models for General Symbolic Reasoning
• Abstract summary: Symbolic-reasoning tasks such as Sudoku, maze pathfinding, and ARC remain challenging for LLMs.<n>We propose R-Qwen, a framework built upon a pretrained Qwen backbone.<n>R-Qwen repeatedly refines a candidate solution through programmatic self-recursion and deep supervision.
• Score: 17.777375944237015
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Hard symbolic-reasoning tasks such as Sudoku, maze pathfinding, and ARC remain challenging for LLMs due to their fixed-depth autoregressive reasoning, which limits systematic search, refinement, and backtracking. While recursive models such as Hierarchical Reasoning Model (HRM) and Tiny Recursive Model (TRM) address this limitation through iterative latent-state refinement, they are typically task-specific and do not leverage pretrained language priors. We propose R-Qwen, a recursive reasoning framework built upon a pretrained Qwen backbone. R-Qwen repeatedly refines a candidate solution through programmatic self-recursion and deep supervision, combining the structured iterative computation of recursive models with the linguistic and reasoning priors of pretrained LLMs. We further adapt Hierarchical Supervision Weighting (HSW) to autoregressive models by exponentially weighting losses across recursive steps. HSW reduces gradient variance by at least 50\%, improves the signal-to-noise ratio of stochastic gradients, and accelerates convergence. Across eight challenging benchmarks, R-Qwen consistently outperforms prior recursive reasoning models and substantially larger LLMs while using a comparable number of trainable parameters. Notably, on ARC-AGI dataset, our model achieves a 27.6\% improvement over the baseline, highlighting the effectiveness of recursive refinement for general symbolic reasoning. These results suggest that recursive reasoning mechanisms and pretrained language model priors are complementary approaches for improving symbolic puzzle-solving. Code and models will be released after acceptance.

Related papers

• Think Deep, Speak Once: Relit, A Recursive Latent Implicit Transformer Framework [2.007262412327553]
  We introduce ReLIT, a hybrid framework for deep, semantically grounded implicit reasoning.<n>We show that ReLIT can be scaled efficiently through recurrent depth rather than parameter width.<n>These results demonstrate that reasoning capability can be scaled efficiently through recurrent depth rather than parameter width.
  arXiv  Detail & Related papers  (2026-08-08T12:50:51Z)
• Recursive Scaling in Masked Diffusion Models [67.93066069475753]
  Masked diffusion models (MDMs) have recently emerged as a promising paradigm for sequence generation.<n>We introduce Recursive Masked Diffusion Models (R-MDMs) to increase model depth without increasing parameter count.<n>Across structured generation tasks, including Sudoku and Countdown, we show that R-MDMs achieve substantially improved parameter efficiency.
  arXiv  Detail & Related papers  (2026-06-16T15:02:49Z)
• Generative Recursive Reasoning [67.22973831501257]
  Generative Recursive reAsoning Models (GRAM) is a framework that turns latent reasoning into probabilistic multi-trajectory.<n>GRAM supports conditional reasoning via $p_(y mid x)$ and, with fixed or absent inputs, unconditional generation via $p_(x)$.
  arXiv  Detail & Related papers  (2026-05-19T05:20:56Z)
• Recursive Multi-Agent Systems [106.82473907888497]
  We introduce RecursiveMAS, a multi-agent framework that casts the entire system as a unified latent-space computation.<n>RecursiveMAS connects heterogeneous agents as a collaboration loop through the lightweight RecursiveLink module.<n> Theoretical analyses of runtime complexity and learning dynamics establish that RecursiveMAS is more efficient than standard text-based MAS.
  arXiv  Detail & Related papers  (2026-04-28T17:59:34Z)
• Recursive Models for Long-Horizon Reasoning [28.82044197167549]
  We show that a model can invoke itself to solve subtasks in isolated contexts.<n>We generalize our framework to modern agentic systems with arbitrary context processing and control flows.
  arXiv  Detail & Related papers  (2026-03-02T17:37:10Z)
• Looping Back to Move Forward: Recursive Transformers for Efficient and Flexible Large Multimodal Models [63.47909317137073]
  Large Multimodal Models (LMMs) have achieved remarkable success in vision-language computation tasks.<n>But their vast parameter counts are often underutilized during both training and inference.<n>We propose RecursiveVLM, a recursive Transformer architecture tailored for LMMs.
  arXiv  Detail & Related papers  (2026-02-09T17:58:23Z)
• Principled RL for Diffusion LLMs Emerges from a Sequence-Level Perspective [85.06838178922791]
  Reinforcement Learning (RL) has proven highly effective for autoregressive language models.<n>But adapting these methods to diffusion large language models (dLLMs) presents fundamental challenges.<n>We propose a principled RL framework that treats entire sequence generation as a single action and uses the ELBO as a tractable sequence-level likelihood proxy.
  arXiv  Detail & Related papers  (2025-12-03T13:05:32Z)
• Adaptive Graph of Thoughts: Test-Time Adaptive Reasoning Unifying Chain, Tree, and Graph Structures [0.0]
  We introduce Adaptive Graph of Thoughts (AGoT), a dynamic, graph-based inference framework.<n>AGoT enhances Large Language Models (LLMs) reasoning solely at test time.<n>We validate our approach on diverse benchmarks spanning multi-hop retrieval, scientific reasoning, and mathematical problem-solving.
  arXiv  Detail & Related papers  (2025-02-07T16:54:19Z)
• Let's reward step by step: Step-Level reward model as the Navigators for Reasoning [64.27898739929734]
  Process-Supervised Reward Model (PRM) furnishes LLMs with step-by-step feedback during the training phase. We propose a greedy search algorithm that employs the step-level feedback from PRM to optimize the reasoning pathways explored by LLMs. To explore the versatility of our approach, we develop a novel method to automatically generate step-level reward dataset for coding tasks and observed similar improved performance in the code generation tasks.
  arXiv  Detail & Related papers  (2023-10-16T05:21:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.