Self-Prophetic Decoding to Unlock Visual Search in LVLMs

• URL: http://arxiv.org/abs/2605.28741v1
• Date: Wed, 27 May 2026 17:01:39 GMT
• Title: Self-Prophetic Decoding to Unlock Visual Search in LVLMs
• Abstract summary: Large Vision-Language Models (LVLMs) are rapidly evolving toward true multimodal reasoning.<n>LVLM visual search faces two key challenges: incompatibility among intrinsic capabilities after post-training, and interference in long multi-step reasoning contexts.<n>We introduce SeProD, a self-prophetic decoding framework that leverages intrinsic single-step capabilities to enable coherent multi-step reasoning.
• Score: 108.77389957341586
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Vision-Language Models (LVLMs) are rapidly evolving toward true multimodal reasoning, with visual search representing a concrete instantiation of the thinking-with-images paradigm. However, LVLM visual search faces two key challenges: incompatibility among intrinsic capabilities after post-training, and interference in long multi-step reasoning contexts. To address these, we identify two novel insights. First, self-regulation between pre- and post-training LVLMs leverages the intrinsic single-step capabilities of the pre-training model to mitigate capability deterioration and long-context interference. Second, probability-based prophetic sampling, replacing naive prompting, provides a probabilistic interface where the pre-training model acts as a prophet and the post-training model selectively accepts prophetic tokens under its output distribution, preserving coherent multi-step reasoning. Building on these insights, we introduce SeProD, a self-prophetic decoding framework that leverages intrinsic single-step capabilities to enable coherent multi-step reasoning in a training-free, plug-and-play manner. Experiments show that SeProD consistently improves multiple visual-search LVLMs across all 12 splits of 4 visual search benchmarks, as well as across general VQA benchmarks, without added computational overhead, thanks to its parallel prophetic acceptance mechanism.

Related papers

• Latent-Aligned Reasoning for Multimodal Recommendation [4.868819130885598]
  We propose LARK, a two-stage latent reasoning framework with complementary alignment mechanisms within a single VLM.<n>In the first stage, learnable latent tokens are interleaved with multi-step chain-of-thought reasoning.<n>In the second stage, latent representations are projected via a bridge and trained with item-to-item contrastive learning.
  arXiv  Detail & Related papers  (2026-09-04T02:24:48Z)
• InterSketch: An Interleaved Reasoning Model with Self-correcting Visual Sketch and Stepwise Reward [24.461407883853344]
  Human-like thought typically involves long-horizon reasoning with an interleaved visual-textual chain-of-thought (VT-CoT)<n>We introduce InterSketch, an interleaved reasoning model to enhance the VT-CoT capability via self-correcting and stepwise reward mechanisms.<n>Experiments on visual reasoning benchmarks demonstrate the effectiveness of InterSketch, even outperforming proprietary models such as Gemini-3-Pro.
  arXiv  Detail & Related papers  (2026-05-26T04:07:49Z)
• Learn to Think: Improving Multimodal Reasoning through Vision-Aware Self-Improvement Training [82.17582358979884]
  Post-training with explicit reasoning traces is common to improve the reasoning capabilities of Multimodal Large Language Models (MLLMs)<n>We propose VISTA, a vision-aware self-improvement training framework for enhancing the multimodal reasoning of MLLMs.
  arXiv  Detail & Related papers  (2026-05-12T10:44:35Z)
• Enhancing Multimodal In-Context Learning via Inductive-Deductive Reasoning [65.15766304205657]
  In-context learning (ICL) allows large models to adapt to tasks using a few examples, yet its extension to vision-language models (VLMs) remains fragile.<n>Our analysis reveals that the fundamental limitation lies in an inductive gap, models often produce correct answers from flawed reasoning.<n>We introduce a framework that restructures multimodal ICL as a principled inductive-deductive process.
  arXiv  Detail & Related papers  (2026-05-04T09:18:19Z)
• SVSR: A Self-Verification and Self-Rectification Paradigm for Multimodal Reasoning [12.51459553492698]
  Self-Verification and Self-Rectification (SVSR) is a unified framework that integrates self-verification and self-rectification into the model's reasoning pipeline.<n>SVSR substantially improves robustness and reliability in complex visual understanding and multimodal reasoning tasks.
  arXiv  Detail & Related papers  (2026-04-11T14:25:17Z)
• Insight-V++: Towards Advanced Long-Chain Visual Reasoning with Multimodal Large Language Models [65.4947731385794]
  We present a unified multi-agent visual reasoning framework that evolves from our foundational image-centric model, Insight-V.<n>We introduce two novel algorithms, ST-GRPO and J-GRPO, which enhance spatial-temporal reasoning and improve evaluative robustness.<n>Experiments on base models like LLaVA-NeXT and Qwen2.5-VL demonstrate significant performance gains across challenging image and video reasoning benchmarks.
  arXiv  Detail & Related papers  (2026-03-18T15:28:07Z)
• Perception-Aware Multimodal Spatial Reasoning from Monocular Images [57.42071289037214]
  spatial reasoning from monocular images is essential for autonomous driving.<n>Current Vision-Language Models (VLMs) still struggle with fine-grained geometric perception.<n>We propose a perception-aware multimodal reasoning framework that equipsVLMs with explicit object-centric grounding ability.
  arXiv  Detail & Related papers  (2026-03-07T02:05:12Z)
• UniT: Unified Multimodal Chain-of-Thought Test-time Scaling [85.590774707406]
  Unified models can handle both multimodal understanding and generation within a single architecture, yet they typically operate in a single pass without iteratively refining their outputs.<n>We introduce UniT, a framework for multimodal test-time scaling that enables a single unified model to reason, verify, and refine across multiple rounds.
  arXiv  Detail & Related papers  (2026-02-12T18:59:49Z)
• CASHEW: Stabilizing Multimodal Reasoning via Iterative Trajectory Aggregation [6.356820150960838]
  We introduce two complementary approaches inspired by test-time scaling to stabilize vision-language models.<n>CASHEW is an inference-time framework that stabilizes reasoning by iteratively aggregating multiple candidate trajectories into higher-quality reasoning traces.<n>CASHEW-RL is trained using Group Sequence Policy Optimization (GSPO) with a composite reward that encourages correct answers grounded in minimal yet sufficient visual evidence.
  arXiv  Detail & Related papers  (2026-01-12T21:24:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.