LaME: Learning to Think in Latent Space for Multimodal Embedding via Information Bottleneck

• URL: http://arxiv.org/abs/2606.13061v1
• Date: Thu, 11 Jun 2026 08:47:25 GMT
• Title: LaME: Learning to Think in Latent Space for Multimodal Embedding via Information Bottleneck
• Abstract summary: We propose LaME, which formulates embedding-oriented latent reasoning as a weakly supervised information bottleneck.<n>LaME employs K learnable reason tokens as a fixed-capacity bottleneck, completing all reasoning within a single forward pass.<n> Experiments on MMEB-v2 and MRMR show that LaME achieves competitive performance, surpassing some explicit CoT-based models.
• Score: 28.83441191878705
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reasoning-driven universal multimodal embedding has advanced rapidly by introducing Chain-of-Thought (CoT) reasoning into the embedding pipeline. Despite the strong performance across both general and complex tasks, this paradigm suffers from two core limitations: (i) autoregressive CoT reasoning incurs high computational cost, making it impractical for low-latency retrieval; and (ii) embedding performance is heavily coupled with CoT annotation quality, making large-scale training unreliable. These raise fundamental questions: Is textual CoT the optimal form of reasoning for embedding, and can effective embedding reasoning be accomplished in latent space? To this end, we propose LaME (Latent Reasoning Multimodal Embedding), which formulates embedding-oriented latent reasoning as a weakly supervised information bottleneck. LaME employs K learnable reason tokens as a fixed-capacity bottleneck, completing all reasoning within a single forward pass. The two weak supervision signals structurally decouple contrastive from autoregressive objectives and eliminate dependence on CoT annotations, while a two-stage training pipeline ensures stable convergence. Experiments on MMEB-v2 and MRMR show that LaME achieves competitive performance, surpassing some explicit CoT-based models, while delivering 60x faster inference than explicit CoT methods and 2x faster than latent baselines with throughput comparable to discriminative embedding models. Code will be released.

Related papers

• SFAD: Speculative Factuality-Aware Decoding [13.350717038599386]
  ConFide is a preference dataset with fine-grained atomic perturbations.<n>Epistemic Friction detects potential hallucinations by quantifying distributional tension weighted by specialist certainty.<n>Asymmetric Logit Steering refines the target distribution through residual-based logit injection.
  arXiv  Detail & Related papers  (2026-09-01T06:46:54Z)
• CONSISTRE: A Unified Consistency-Aware Framework for Document-Level Relation Extraction with Large Language Models [11.083514956613383]
  Document-level relation extraction (DocRE) aims to extract relations among multiple entities across extended contexts.<n>Large language models (LLMs) show remarkable reasoning capabilities in information extraction.<n>LLMs may violate fundamental relational constraints such as transitivity, symmetry, and functional uniqueness, leading to contradictory and unreliable outputs.<n>We propose a unified consistency-aware framework for DocRE that addresses this limitation through two complementary tracks.
  arXiv  Detail & Related papers  (2026-07-27T11:56:29Z)
• SuCo: Sufficiency-guided Continuous Adaptive Reasoning [59.17222918775686]
  Large Reasoning Models (LRMs) often generate excessively long Chain-of-Thoughts (CoT)<n>Existing efforts to mitigate this inefficiency typically rely on discrete reasoning modes or fixed budget tiers.<n>We introduce Minimal Sufficient CoT (MSC), defined as the shortest prefix of a CoT trajectory which is adequate for producing the correct answer.
  arXiv  Detail & Related papers  (2026-06-16T08:52:15Z)
• PLUME: Latent Reasoning Based Universal Multimodal Embedding [52.35354073629127]
  Universal multimodal embedding (UME) maps heterogeneous inputs into a shared retrieval space with a single model.<n>Recent approaches improve UME by generating explicit chain-of-thought (CoT) rationales before extracting embeddings.<n>We propose PLUME, a latent reasoning framework that advances UME by replacing verbalized CoT with a short autoregressive rollout of continuous latent states.
  arXiv  Detail & Related papers  (2026-04-02T14:04:53Z)
• Reinforcing Structured Chain-of-Thought for Video Understanding [49.3086326382764]
  We introduce Summary-Driven Reinforcement Learning (SDRL)<n>SDRL is a single-stage RL framework that obviates the need forSupervised Fine-Tuning (SFT)<n>Our method achieves state-of-the-art performance on seven public VideoQA datasets.
  arXiv  Detail & Related papers  (2026-03-26T22:11:14Z)
• Embed-RL: Reinforcement Learning for Reasoning-Driven Multimodal Embeddings [44.77164359074224]
  Multimodal Large Language Models (MLLMs) have become pivotal for advancing Universal Multimodal Embeddings (UME)<n>Recent studies demonstrate that incorporating generative Chain-of-Thought (CoT) reasoning can substantially enhance task-specific representations.<n>We propose a reasoning-driven UME framework that integrates Embedder-Guided Reinforcement Learning (EG-RL) to optimize the Reasoner to produce evidential Traceability CoT.
  arXiv  Detail & Related papers  (2026-02-14T15:35:03Z)
• Efficient Paths and Dense Rewards: Probabilistic Flow Reasoning for Large Language Models [37.37112282368718]
  CoT-Flow is a framework that reconceptualizes discrete reasoning steps as a continuous probabilistic flow.<n>We show that CoT-Flow achieves a superior balance between inference efficiency and reasoning performance.
  arXiv  Detail & Related papers  (2026-01-14T07:52:05Z)
• SIM-CoT: Supervised Implicit Chain-of-Thought [108.30049193668083]
  Implicit Chain-of-Thought (CoT) methods offer a token-efficient alternative to explicit CoT reasoning in Large Language Models.<n>We identify a core latent instability issue when scaling the computational budget of implicit CoT.<n>We propose SIM-CoT, a plug-and-play training module that introduces step-level supervision to stabilize and enrich the latent reasoning space.
  arXiv  Detail & Related papers  (2025-09-24T17:01:32Z)
• Reinforced Latent Reasoning for LLM-based Recommendation [92.56166822197919]
  Large Language Models (LLMs) have demonstrated impressive reasoning capabilities in complex problem-solving tasks.<n>Existing methods typically rely on fine-tuning with explicit chain-of-thought (CoT) data.<n>In this work, we explore an alternative approach that shifts from explicit CoT reasoning to compact, information-dense latent reasoning.
  arXiv  Detail & Related papers  (2025-05-25T11:03:45Z)
• Fractured Chain-of-Thought Reasoning [61.647243580650446]
  We introduce Fractured Sampling, a unified inference-time strategy that interpolates between full CoT and solution-only sampling.<n>We show that Fractured Sampling consistently achieves superior accuracy-cost trade-offs, yielding steep log-linear scaling gains in Pass@k versus token budget.
  arXiv  Detail & Related papers  (2025-05-19T11:30:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.