GLaQ: Grounding Latent Queries in Visual Evidence for Multimodal Reasoning

• URL: http://arxiv.org/abs/2608.15517v2
• Date: Tue, 18 Aug 2026 04:25:28 GMT
• Title: GLaQ: Grounding Latent Queries in Visual Evidence for Multimodal Reasoning
• Abstract summary: Chain-of-thought reasoning has substantially improved the problem-solving capabilities of multimodal large language models.<n>Fine-grained visual evidence, however, remains difficult to preserve and reuse across text-based reasoning steps.<n>We introduce GLaQ, a grounded latent-query framework that replaces sequential latent rollout with a fixed set of context-conditioned queries.
• Score: 27.232897918541067
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Chain-of-thought reasoning has substantially improved the problem-solving capabilities of multimodal large language models. Fine-grained visual evidence, however, remains difficult to preserve and reuse across text-based reasoning steps. To address this limitation, tool-augmented thinking-with-images methods maintain visual access externally by revisiting or manipulating the image, but require predefined tools and additional inference-time processing. As an internal alternative, continuous visual latent reasoning retains intermediate computation in hidden states. However, its prevailing autoregressive construction makes each latent state depend on its predecessors, so later states may repeat information already present in the latent sequence rather than capture complementary visual details. We introduce GLaQ, a grounded latent-query framework that replaces sequential latent rollout with a fixed set of context-conditioned queries grounded in the original visual tokens. The grounded queries are reinjected for answer generation, providing direct and coordinated access to source visual evidence. We train GLaQ with localized-view supervision followed by reinforcement learning under task-level rewards. Across five benchmarks for fine-grained visual understanding and perception, GLaQ-7B gains 5.99--9.66\% over its base model and leads all compared visual latent methods, suggesting that direct query-to-image grounding can recover localized evidence from the full image without external visual operations or autoregressive latent rollouts.

Related papers

• Navigating Sparse Evidence: Agentic Visual RAG via Explicit Context Selection and Consolidation [50.92223196290564]
  We propose a unified agent loop for explicit evidence selection and consolidation.<n>During exploration, SCoRE retains only query-relevant observations and their source pointers in a maintained textual ledger.<n>At termination, it reloads the referenced original images and consolidates the visual evidence for answering, arranging it into a logical sequence.
  arXiv  Detail & Related papers  (2026-09-14T16:11:35Z)
• REVEAL: A Rubric-Guided Agent for Explicit Evidence Sufficiency Verificationin Long-Video Question Answering [25.519927761448603]
  REVEAL is a rubric-guided agent framework for long-video question answering.<n>We introduce an adaptive visual-similarity-based preprocessing pipeline that groups visually coherent adjacent frames into natural event units.<n>Built upon this structured memory, REVEAL uses an automatically constructed rubric library to explicitly verify whether retrieved evidence satisfies sufficiency criteria.
  arXiv  Detail & Related papers  (2026-08-09T09:55:42Z)
• Remember-R1: Mitigating Long-Context Visual Forgetting through Reinforcement Learning [50.56518447675037]
  Multimodal large language models (MLLMs) increasingly rely on long chain-of-thought reasoning for complex tasks.<n>As reasoning sequences lengthen, models may gradually rely less on visual evidence and more on accumulated textual context, leading to visual forgetting.<n>We propose Remember-R1, a reinforcement learning framework that mitigates long-context visual forgetting by applying process-level supervision directly on the original reasoning trajectory.
  arXiv  Detail & Related papers  (2026-08-02T15:31:31Z)
• Be Faithful When Response: Returning Fluent and Grounded Answers for Vision-Language Models Reinforcement Learning [12.622982203412478]
  Reinforcement Learning (RL) is an important paradigm for improving the reasoning capabilities of Vision-Language Models (VLMs)<n>We propose a Faithful Warm-Start (FWS) strategy that first curates samples with explicit vision-language causal relationships from six general VQA benchmarks.<n>A VLM-based judge is employed to further purify the dataset, ensuring strong causal consistency and visual faithfulness.<n> Experimental results show that such faithful supervision improves answer accuracy, stabilizes RL training, and reduces visually unsupported reasoning.
  arXiv  Detail & Related papers  (2026-06-29T08:58:33Z)
• Diagnosing Visual Ignorance in Vision-Language Models [29.2851901986069]
  Vision-Language Models (VLMs) frequently rely on language priors, producing confident answers that are weakly grounded in visual evidence.<n>In this work, we study language-prior reliance from both mechanistic and behavioral perspectives.<n>We find that language-prior reliance is a systematic routing failure affecting both model internals and benchmark validity.
  arXiv  Detail & Related papers  (2026-06-05T04:16:01Z)
• ATLAS: Agentic or Latent Visual Reasoning? One Word is Enough for Both [55.182037225013836]
  ATLAS is a framework in which a single discrete 'word', termed as a functional token, serves both as an agentic operation and a latent visual reasoning unit.<n> ATLAS achieves superior performance on challenging benchmarks while maintaining clear interpretability.
  arXiv  Detail & Related papers  (2026-05-14T17:59:55Z)
• Entropy-Gradient Grounding: Training-Free Evidence Retrieval in Vision-Language Models [77.3748853516374]
  We propose a training-free, model-intrinsic grounding method for vision-language models.<n>We compute the entropy of the model's next-token distribution and backpropagate it to the visual token embeddings to obtain an entropy-gradient relevance map.<n>We then extract and rank multiple coherent regions to support multi-evidence queries, and introduce an iterative zoom-and-reground procedure with a spatial-entropy stopping rule.
  arXiv  Detail & Related papers  (2026-04-09T16:51:42Z)
• Self-Corrected Image Generation with Explainable Latent Rewards [55.29175717238288]
  We propose xLARD, a self-correcting framework that guides generation through Explainable LAtent RewarDs.<n>xLARD introduces a lightweight corrector that refines latent representations based on structured feedback from model-generated references.<n> Experiments show that xLARD improves semantic alignment and visual fidelity while maintaining generative priors.
  arXiv  Detail & Related papers  (2026-03-26T02:59:35Z)
• Improving Visual Reasoning with Iterative Evidence Refinement [41.18410988040976]
  Vision language models (VLMs) are increasingly capable of reasoning over images.<n>But robust visual reasoning often requires re-grounding intermediate steps in the underlying visual evidence.<n>We propose an end-to-end self-revisit framework, SIEVE, that trains models to re-engage image evidence through internal representations.
  arXiv  Detail & Related papers  (2026-03-14T21:08:21Z)
• ReAG: Reasoning-Augmented Generation for Knowledge-based Visual Question Answering [54.72902502486611]
  ReAG is a Reasoning-Augmented Multimodal RAG approach that combines coarse- and fine-grained retrieval with a critic model that filters irrelevant passages.<n>ReAG significantly outperforms prior methods, improving answer accuracy and providing interpretable reasoning grounded in retrieved evidence.
  arXiv  Detail & Related papers  (2025-11-27T19:01:02Z)
• Look As You Think: Unifying Reasoning and Visual Evidence Attribution for Verifiable Document RAG via Reinforcement Learning [55.232400251303794]
  Look As You Think (LAT) is a reinforcement learning framework that trains models to produce verifiable reasoning paths with consistent attribution.<n>LAT consistently improves the vanilla model in both single- and multi-image settings, yielding average gains of 8.23% in soft exact match (EM) and 47.0% in IoU@0.5.
  arXiv  Detail & Related papers  (2025-11-15T02:50:23Z)
• Shifting More Attention to Visual Backbone: Query-modulated Refinement Networks for End-to-End Visual Grounding [35.44496191453257]
  Existing methods use pre-trained query-agnostic visual backbones to extract visual feature maps independently. We argue that the visual features extracted from the visual backbones and the features needed for multimodal reasoning are inconsistent. We propose a Query-modulated Refinement Network (QRNet) to address the inconsistent issue.
  arXiv  Detail & Related papers  (2022-03-29T11:17:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.