LanteRn: Latent Visual Structured Reasoning

• URL: http://arxiv.org/abs/2603.25629v1
• Date: Thu, 26 Mar 2026 16:41:59 GMT
• Title: LanteRn: Latent Visual Structured Reasoning
• Abstract summary: We introduce LanteRn, a framework that enables visual reasoning to occur directly in latent space.<n>LanteRn augments a vision-language transformer with the ability to generate and attend to continuous visual thought embeddings during inference.<n>We evaluate LanteRn on three perception-centric benchmarks (VisCoT, V*, and Blink)
• Score: 7.141402207573525
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While language reasoning models excel in many tasks, visual reasoning remains challenging for current large multimodal models (LMMs). As a result, most LMMs default to verbalizing perceptual content into text, a strong limitation for tasks requiring fine-grained spatial and visual understanding. While recent approaches take steps toward thinking with images by invoking tools or generating intermediate images, they either rely on external modules, or incur unnecessary computation by reasoning directly in pixel space. In this paper, we introduce LanteRn, a framework that enables LMMs to interleave language with compact latent visual representations, allowing visual reasoning to occur directly in latent space. LanteRn augments a vision-language transformer with the ability to generate and attend to continuous visual thought embeddings during inference. We train the model in two stages: supervised fine-tuning to ground visual features in latent states, followed by reinforcement learning to align latent reasoning with task-level utility. We evaluate LanteRn on three perception-centric benchmarks (VisCoT, V*, and Blink), observing consistent improvements in visual grounding and fine-grained reasoning. These results suggest that internal latent representations provide a promising direction for more efficient multimodal reasoning.

Related papers

• Reasoning with Image Generation [18.95650438552521]
  Chain-of-thought reasoning has revolutionized natural language processing by enabling large language models to decompose problems into intermediate steps before answering.<n>We propose ReImaGin, which leverages image generation models as a flexible visual reasoning mechanism for multimodal LLMs.<n>We show that ReImaGin consistently outperforms both text-only reasoning and specialist vision-tool baselines, with gains of up to 25%.
  arXiv  Detail & Related papers  (2026-09-14T22:29:03Z)
• Vision Inference Former: Sustaining Visual Consistency in Multimodal Large Language Models [58.9387276447485]
  Vision Inference Former (VIF) is a lightweight architectural module that establishes a bridge between pure visual representations and the model's output space.<n>We conduct experiments on 14 benchmark tasks covering general reasoning, OCR, table understanding, vision-centric evaluation, and hallucination.
  arXiv  Detail & Related papers  (2026-05-18T10:04:22Z)
• UniVLR: Unifying Text and Vision in Visual Latent Reasoning for Multimodal LLMs [36.74593429466411]
  We propose a unified visual latent reasoning framework that treats textual reasoning and auxiliary visual evidence as a shared visual workspace.<n>We show that UniVLR outperforms prior visual latent reasoning methods while using substantially fewer generated reasoning tokens.
  arXiv  Detail & Related papers  (2026-05-12T09:40:03Z)
• Latent Implicit Visual Reasoning [59.39913238320798]
  We propose a task-agnostic mechanism that trains LMMs to discover and use visual reasoning tokens without explicit supervision.<n>Our approach outperforms direct fine-tuning and achieves state-of-the-art results on a diverse range of vision-centric tasks.
  arXiv  Detail & Related papers  (2025-12-24T14:59:49Z)
• Monet: Reasoning in Latent Visual Space Beyond Images and Language [55.424507246294326]
  "Thinking with images" has emerged as an effective paradigm for advancing visual reasoning.<n>Existing methods fall short of human-like abstract visual thinking.<n>We introduce Monet, a training framework that enables multimodal large language models to reason directly within the latent visual space.
  arXiv  Detail & Related papers  (2025-11-26T13:46:39Z)
• Visual Jigsaw Post-Training Improves MLLMs [58.29961336087896]
  We introduce Visual Jigsaw, a generic self-supervised post-training framework designed to strengthen visual understanding in large language models (MLLMs)<n>Visual Jigsaw is formulated as a general ordering task: visual inputs are partitioned, shuffled, and the model must reconstruct the visual information by producing the correct permutation in natural language.<n>Extensive experiments demonstrate substantial improvements in fine-grained perception, temporal reasoning, and 3D spatial understanding.
  arXiv  Detail & Related papers  (2025-09-29T17:59:57Z)
• Latent Visual Reasoning [40.347006722601975]
  We introduce Latent Visual Reasoning (LVR), a new paradigm that enables autoregressive reasoning directly in the visual embedding space.<n>We show that LVR substantially improves fine-grained visual understanding and perception, achieving 71.67% on MMVP compared to 66.67% with Qwen2.5-VL.
  arXiv  Detail & Related papers  (2025-09-29T03:52:01Z)
• Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawing [62.447497430479174]
  Drawing to reason in space is a novel paradigm that enables LVLMs to reason through elementary drawing operations in the visual space.<n>Our model, named VILASR, consistently outperforms existing methods across diverse spatial reasoning benchmarks.
  arXiv  Detail & Related papers  (2025-06-11T17:41:50Z)
• Cantor: Inspiring Multimodal Chain-of-Thought of MLLM [83.6663322930814]
  We argue that converging visual context acquisition and logical reasoning is pivotal for tackling visual reasoning tasks. We propose an innovative multimodal CoT framework, termed Cantor, characterized by a perception-decision architecture. Our experiments demonstrate the efficacy of the proposed framework, showing significant improvements in multimodal CoT performance.
  arXiv  Detail & Related papers  (2024-04-24T17:59:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.