DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning

• URL: http://arxiv.org/abs/2606.15160v1
• Date: Sat, 13 Jun 2026 07:12:46 GMT
• Title: DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
• Abstract summary: We present DLWM, a multimodal reasoning framework that combines latent-space reasoning with reinforcement learning.<n>First, we construct a set of diverse latent world hypotheses in continuous latent space, each capturing a different plausible interpretation of the visual input.<n>Second, we formulate the latent reasoning process as a resource-constrained sequential decision problem.
• Score: 5.070905256010651
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reasoning capabilities of multimodal large language models (MLLMs) have improved considerably in recent years. Existing approaches typically rely on explicit chain-of-thought or continuous latent-space trajectories to enhance multi-step reasoning. However, these methods generally assume that an input admits a single latent interpretation and unfold reasoning along a fixed path or under a uniform computation budget. In real-world multimodal settings, visual observations are often subject to occlusion, blur, viewpoint variation, or semantic ambiguity, giving rise to multiple plausible interpretations. A uniform reasoning strategy not only limits the model's ability to explore multiple hypotheses but also incurs high memory usage and rollout cost. We present DLWM (Diverse Latent World Models), a multimodal reasoning framework that combines latent-space reasoning with reinforcement learning. First, we construct a set of diverse latent world hypotheses in continuous latent space, each capturing a different plausible interpretation of the visual input, and unfold latent reasoning independently on each hypothesis. An orthogonality-based diversity regularizer explicitly prevents hypothesis collapse. Second, we formulate the latent reasoning process as a resource-constrained sequential decision problem and introduce a resource-aware reinforcement learning policy that adaptively allocates computation across hypotheses, dynamically deciding whether to expand, terminate, or merge reasoning paths, thereby substantially reducing memory footprint and improving rollout efficiency. Experiments on multiple multimodal reasoning benchmarks demonstrate that DLWM outperforms existing methods by 2-5 points in accuracy while reducing memory usage by 24%.

Related papers

• Walk the Talk: Bridging the Reasoning-Action Gap for Thinking with Images via Multimodal Agentic Policy Optimization [89.68681087743876]
  Multimodal Large Language Models (MLLMs) have incentivized models to think with images'' by actively invoking visual tools during multi-turn reasoning.<n>The common Reinforcement Learning practice of relying on outcome-based rewards ignores the fact that textual plausibility often masks executive failure.<n>We introduce Multimodal Agentic Policy Optimization (MAPO), bridging the gap between textual reasoning and visual actions generated by models within their Multimodal Chain-of-Thought.
  arXiv  Detail & Related papers  (2026-04-08T07:48:07Z)
• PLUME: Latent Reasoning Based Universal Multimodal Embedding [52.35354073629127]
  Universal multimodal embedding (UME) maps heterogeneous inputs into a shared retrieval space with a single model.<n>Recent approaches improve UME by generating explicit chain-of-thought (CoT) rationales before extracting embeddings.<n>We propose PLUME, a latent reasoning framework that advances UME by replacing verbalized CoT with a short autoregressive rollout of continuous latent states.
  arXiv  Detail & Related papers  (2026-04-02T14:04:53Z)
• Thinking in Uncertainty: Mitigating Hallucinations in MLRMs with Latent Entropy-Aware Decoding [38.5840117402958]
  We argue that adequate contextual reasoning information can be directly extracted from the token probability distribution.<n>We present Latent Entropy-Aware Decoding, an efficient plug-and-play decoding strategy.<n>The model employs probability-weighted continuous embeddings under high-entropy states and transitions back to discrete token embeddings as entropy decreases.
  arXiv  Detail & Related papers  (2026-03-09T12:47:54Z)
• MIND: Multi-rationale INtegrated Discriminative Reasoning Framework for Multi-modal Large Models [15.860796863065737]
  We propose a Multi-rationale INtegrated Discriminative (MIND) reasoning framework.<n>It is designed to endow MLLMs with human-like cognitive abilities of "Understand -> Rethink -> Correct"<n>It achieves a paradigm evolution from passive imitation-based reasoning to active discriminative reasoning.
  arXiv  Detail & Related papers  (2025-12-05T08:41:44Z)
• Multi-Path Collaborative Reasoning via Reinforcement Learning [54.8518809800168]
  Chain-of-Thought (CoT) reasoning has significantly advanced the problem-solving capabilities of Large Language Models (LLMs)<n>Recent methods attempt to address this by generating soft abstract tokens to enable reasoning in a continuous semantic space.<n>We propose Multi-Path Perception Policy Optimization (M3PO), a novel reinforcement learning framework that explicitly injects collective insights into the reasoning process.
  arXiv  Detail & Related papers  (2025-12-01T10:05:46Z)
• PA-FAS: Towards Interpretable and Generalizable Multimodal Face Anti-Spoofing via Path-Augmented Reinforcement Learning [42.24912525813944]
  Face anti-spoofing (FAS) has recently advanced in multimodal fusion, cross-domain generalization, and interpretability.<n>We propose PA-FAS, which enhances reasoning paths by constructing high-quality extended reasoning sequences from limited annotations.<n>We also introduce an answer-shuffling mechanism during SFT to force comprehensive multimodal analysis instead of using superficial cues.
  arXiv  Detail & Related papers  (2025-11-22T05:55:08Z)
• Directional Reasoning Injection for Fine-Tuning MLLMs [51.53222423215055]
  Multimodal large language models (MLLMs) are rapidly advancing, yet their reasoning ability often lags behind that of strong text-only counterparts.<n>Existing methods to bridge this gap rely on supervised fine-tuning over large-scale multimodal reasoning data or reinforcement learning.<n>We propose Directional Reasoning Injection for Fine-Tuning (DRIFT) to solve this problem.
  arXiv  Detail & Related papers  (2025-10-16T18:06:46Z)
• A Survey on Latent Reasoning [100.54120559169735]
  Large Language Models (LLMs) have demonstrated impressive reasoning capabilities.<n>CoT reasoning that verbalizes intermediate steps limits the model's expressive bandwidth.<n>Latent reasoning tackles this bottleneck by performing multi-step inference entirely in the model's continuous hidden state.
  arXiv  Detail & Related papers  (2025-07-08T17:29:07Z)
• Beyond DAGs: A Latent Partial Causal Model for Multimodal Learning [80.44084021062105]
  We propose a novel latent partial causal model for multimodal data, featuring two latent coupled variables, connected by an undirected edge, to represent the transfer of knowledge across modalities.<n>Under specific statistical assumptions, we establish an identifiability result, demonstrating that representations learned by multimodal contrastive learning correspond to the latent coupled variables up to a trivial transformation.<n>Experiments on a pre-trained CLIP model embodies disentangled representations, enabling few-shot learning and improving domain generalization across diverse real-world datasets.
  arXiv  Detail & Related papers  (2024-02-09T07:18:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.