LatentUM: Unleashing the Potential of Interleaved Cross-Modal Reasoning via a Latent-Space Unified Model

• URL: http://arxiv.org/abs/2604.02097v1
• Date: Thu, 02 Apr 2026 14:22:29 GMT
• Title: LatentUM: Unleashing the Potential of Interleaved Cross-Modal Reasoning via a Latent-Space Unified Model
• Abstract summary: We introduce LatentUM, a novel unified model that represents all modalities within a shared semantic latent space.<n>LatentUM substantially alleviates bias and strengthens cross-modal alignment, allowing it to achieve state-of-the-art performance on the Visual Spatial Planning benchmark.
• Score: 47.486827808818184
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unified models (UMs) hold promise for their ability to understand and generate content across heterogeneous modalities. Compared to merely generating visual content, the use of UMs for interleaved cross-modal reasoning is more promising and valuable, e.g., for solving understanding problems that require dense visual thinking, improving visual generation through self-reflection, or modeling visual dynamics of the physical world guided by stepwise action interventions. However, existing UMs necessitate pixel decoding as a bridge due to their disjoint visual representations for understanding and generation, which is both ineffective and inefficient. In this paper, we introduce LatentUM, a novel unified model that represents all modalities within a shared semantic latent space, eliminating the need for pixel-space mediation between visual understanding and generation. This design naturally enables flexible interleaved cross-modal reasoning and generation. Beyond improved computational efficiency, the shared representation substantially alleviates codec bias and strengthens cross-modal alignment, allowing LatentUM to achieve state-of-the-art performance on the Visual Spatial Planning benchmark, push the limits of visual generation through self-reflection, and support world modeling by predicting future visual states within the shared semantic latent space.

Related papers

• Steering Visual Generation in Unified Multimodal Models with Understanding Supervision [42.765106450407814]
  Unified multimodal models are envisioned to bridge the gap between understanding and generation.<n>We propose Understanding-Oriented Post-Training (UNO), a lightweight framework that treats understanding not only as a distinct task, but also a direct supervisory signal to steer generative representations.
  arXiv  Detail & Related papers  (2026-05-07T07:20:04Z)
• Free Lunch for Unified Multimodal Models: Enhancing Generation via Reflective Rectification with Inherent Understanding [20.397510070808238]
  Unified Multimodal Models (UMMs) aim to integrate visual understanding and generation within a single structure.<n>UMMs exhibit a notable capability mismatch, where their understanding capability significantly outperforms their generation.<n>We propose UniRect-CoT, a training-free unified rectification chain-of-thought framework.
  arXiv  Detail & Related papers  (2026-04-15T06:41:56Z)
• UniCom: Unified Multimodal Modeling via Compressed Continuous Semantic Representations [45.861324782424326]
  Current unified multimodal models typically rely on discrete visual tokenizers to bridge the modality gap.<n>We introduce UniCom, a unified framework that harmonizes multimodal understanding and generation via compressed continuous representation.
  arXiv  Detail & Related papers  (2026-03-11T12:14:26Z)
• Perception-Aware Multimodal Spatial Reasoning from Monocular Images [57.42071289037214]
  spatial reasoning from monocular images is essential for autonomous driving.<n>Current Vision-Language Models (VLMs) still struggle with fine-grained geometric perception.<n>We propose a perception-aware multimodal reasoning framework that equipsVLMs with explicit object-centric grounding ability.
  arXiv  Detail & Related papers  (2026-03-07T02:05:12Z)
• Multimodal Latent Reasoning via Hierarchical Visual Cues Injection [16.779425236020433]
  This work posits that robust reasoning should evolve within a latent space, integrating multimodal signals seamlessly.<n>We propose a novel framework that instills deliberate, "slow thinking" without depending on superficial textual rationales.<n>We show that test-time scaling is effective when incorporating vision knowledge, and that integrating hierarchical information significantly enhances the model's understanding of complex scenes.
  arXiv  Detail & Related papers  (2026-02-05T06:31:12Z)
• MentisOculi: Revealing the Limits of Reasoning with Mental Imagery [63.285794947638614]
  We develop MentisOculi, a suite of multi-step reasoning problems amenable to visual solution.<n> evaluating visual strategies ranging from latent tokens to explicit generated imagery, we find they generally fail to improve performance.<n>Our findings suggest that despite their inherent appeal, visual thoughts do not yet benefit model reasoning.
  arXiv  Detail & Related papers  (2026-02-02T18:49:06Z)
• Interleaved Latent Visual Reasoning with Selective Perceptual Modeling [42.93438443502933]
  Interleaved reasoning paradigms enhance Multimodal Large Language Models (MLLMs) with visual feedback but are hindered by the prohibitive computational cost.<n>A promising alternative, latent visual reasoning, circumvents this bottleneck yet currently forces a critical trade-off.<n>We introduce Interleaved Latent Visual Reasoning (ILVR), a framework that unifies dynamic state evolution with precise perceptual modeling.
  arXiv  Detail & Related papers  (2025-12-05T12:09:39Z)
• Visual Bridge: Universal Visual Perception Representations Generating [27.034175361589572]
  We propose a universal visual perception framework based on flow matching that can generate diverse visual representations across multiple tasks.<n>Our approach formulates the process as a universal flow-matching problem from image patch tokens to task-specific representations.<n>Our model achieves competitive performance in both zero-shot and fine-tuned settings, outperforming prior generalist and several specialist models.
  arXiv  Detail & Related papers  (2025-11-11T06:25:30Z)
• Ming-UniVision: Joint Image Understanding and Generation with a Unified Continuous Tokenizer [50.69959748410398]
  We introduce MingTok, a new family of visual tokenizers with a continuous latent space for unified autoregressive generation and understanding.<n>MingTok adopts a three-stage sequential architecture involving low-level encoding, semantic expansion, and visual reconstruction.<n>Built on top of it, Ming-UniVision eliminates the need for task-specific visual representations, and unifies diverse vision-language tasks under a single autoregrsssive prediction paradigm.
  arXiv  Detail & Related papers  (2025-10-08T02:50:14Z)
• Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawing [62.447497430479174]
  Drawing to reason in space is a novel paradigm that enables LVLMs to reason through elementary drawing operations in the visual space.<n>Our model, named VILASR, consistently outperforms existing methods across diverse spatial reasoning benchmarks.
  arXiv  Detail & Related papers  (2025-06-11T17:41:50Z)
• Multi-granularity Contrastive Cross-modal Collaborative Generation for End-to-End Long-term Video Question Answering [53.39158264785098]
  Long-term Video Question Answering (VideoQA) is a challenging vision-and-language bridging task. We present an entirely end-to-end solution for VideoQA: Multi-granularity Contrastive cross-modal collaborative Generation model.
  arXiv  Detail & Related papers  (2024-10-12T06:21:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.