Leveraging Latent Visual Reasoning in Silence

• URL: http://arxiv.org/abs/2605.18641v1
• Date: Mon, 18 May 2026 16:46:02 GMT
• Title: Leveraging Latent Visual Reasoning in Silence
• Abstract summary: We show that replacing latent tokens with random noise or removing them completely causes little performance degradation across spatial reasoning benchmarks.<n>We propose an attention-based reward that encourages generated latent tokens to interact with later text tokens during RL.
• Score: 46.71750408786006
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Latent visual reasoning involves visual evidence more directly in multimodal reasoning by inserting continuous latent tokens before textual generation. However, the necessity of these latent tokens at inference remains ambiguous. We show that replacing latent tokens with random noise or removing them completely causes little performance degradation across spatial reasoning benchmarks. Reinforcement learning further diminishes the latent generation behavior after post-training. These observations raise a central question: Is latent visual reasoning still meaningful? We argue that its value should be measured by how effectively latent tokens guide learning, rather than whether they persist as an inference-time format. Our analysis shows that latent reasoning is unevenly favorable across question types, yet hard task-level routing for applying latent generation is brittle. Motivated by these findings, we propose an attention-based reward that encourages generated latent tokens to interact with later text tokens during RL. This reward promotes latent utilization when the latent mode is activated while preserving the flexibility to use pure-text reasoning. Experiments show that our method improves performance across perception and visual reasoning benchmarks, even when latent tokens are rarely generated after post-training. Our results highlight that, without explicit expression at inference, latent visual reasoning can shape better visual grounding and more accurate textual reasoning in silence. Our code and trained models are publicly available at \href{https://github.com/ddydyd32/silent-lvr/tree/master}{GitHub} and \href{https://huggingface.co/collections/cornuHGF/silent-lvr}{Hugging Face}.

Related papers

• Why Struggle with Continuous Latents? Interpretable Discrete Latent Reasoning via Rendered Compression [42.15219720606396]
  Large language models achieve high reasoning performance via explicit chain-of-thought and reinforcement learning.<n>Latent reasoning reduces this cost by shifting computation into a latent space.<n>We propose textbfDiscrete Latent Reasoning(DLR), the first method that converts continuous latent states into explicit discrete tokens.
  arXiv  Detail & Related papers  (2026-06-29T02:34:52Z)
• What is Holding Back Latent Visual Reasoning? [23.63938540988447]
  We find that model accuracy is unaffected when latent tokens are replaced by uninformative dummy'' tokens.<n>Our experiments reveal two crucial issues holding back latent visual reasoning.
  arXiv  Detail & Related papers  (2026-05-18T14:14:49Z)
• Self-Consistent Latent Reasoning: Long Latent Sequence Reasoning for Vision-Language Model [56.21523258053447]
  SCOLAR (Self-COnsistent LAtent Reasoning) introduces a lightweight detransformer that generates auxiliary visual tokens in a single shot.<n>SCOLAR extends acceptable latent CoT length by over $30times$, achieves state-of-the-art among open-source models on real-world reasoning benchmarks.
  arXiv  Detail & Related papers  (2026-05-12T14:13:08Z)
• Evading Visual Aphasia: Contrastive Adaptive Semantic Token Pruning for Vision-Language Models [52.78477729846771]
  We introduce COAST (COntrastive Adaptive Semantic Token Pruning), a training-free pruning framework that casts compression as adaptive semantic routing.<n> COAST consistently outperforms strong pruning baselines across token budgets and generalizes across multiple LVLM families.
  arXiv  Detail & Related papers  (2026-05-10T09:07:04Z)
• Visual Latents Know More Than They Say: Unsilencing Latent Reasoning in MLLMs [54.16324124242172]
  Continuous latent-space reasoning offers a compact alternative to textual chain-of-thought for multimodal models.<n>We identify a previously overlooked optimization pathology in existing latent visual reasoning methods.<n>We show that inference-time latent optimization, without any parameter updates, effectively unleashes the suppressed reasoning capacity of visual latents.
  arXiv  Detail & Related papers  (2026-05-04T15:36:12Z)
• Imagination Helps Visual Reasoning, But Not Yet in Latent Space [65.80396132375571]
  We investigate the validity of latent reasoning using Causal Mediation Analysis.<n>We show that latent tokens encode limited visual information and exhibit high similarity.<n>We propose a straightforward alternative named CapImagine, which teaches the model to explicitly imagine using text.
  arXiv  Detail & Related papers  (2026-02-26T08:56:23Z)
• Latent Reasoning in LLMs as a Vocabulary-Space Superposition [80.01651003144282]
  Large language models (LLMs) demonstrate strong reasoning abilities with chain-of-thought prompting, but explicit reasoning introduces substantial computational overhead.<n>Recent work on latent reasoning reduces this cost by reasoning in latent space without explicit supervision, but performance drops significantly.<n>To address this, we restrict the latent space to the column space of the LLM vocabulary, treating latent reasoning as a superposition over vocabulary probabilities.<n>Once latent reasoning concludes, it collapses into an eigenstate of explicit reasoning to yield the final answer.<n>Latent-SFT sets a new state of the art on GSM8k, matching explicit
  arXiv  Detail & Related papers  (2025-10-17T10:51:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.