Dual-Latent Memory Routing for Vision-Language Reasoning

• URL: http://arxiv.org/abs/2609.05539v1
• Date: Wed, 02 Sep 2026 10:45:25 GMT
• Title: Dual-Latent Memory Routing for Vision-Language Reasoning
• Abstract summary: Multimodal large language models (MLLMs) have recently made strong progress in vision-language reasoning, yet their performance often degrades as generations grow longer.<n>Inspired by how humans separately recall what they see and what they infer when solving complex tasks, we propose DLMR.<n>It equips MLLMs with Dual Latent Memories: a visual memory that compresses image evidence and a reasoning memory that tracks intermediate conclusions and constraints.
• Score: 39.90357280328134
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal large language models (MLLMs) have recently made strong progress in vision-language reasoning, yet their performance often degrades as generations grow longer. A key factor is that they frequently lose track of earlier visual evidence and intermediate constraints under a monolithic growing context. Inspired by how humans separately recall what they see and what they infer when solving complex tasks, we propose DLMR, a parameter-efficient mechanism that equips MLLMs with Dual Latent Memories: a visual memory that compresses image evidence and a reasoning memory that tracks intermediate conclusions and constraints. A Router then dynamically decides which memory and how much to reuse during inference, preserving visual grounding while maintaining coherent long-horizon reasoning. DLMR is trained in three stages, from latent memory construction to selective router learning, while keeping the base MLLM frozen, yielding substantial gains on both general and reasoning benchmarks with only a small number of additional trainable parameters. Analyses further show interpretable, state-dependent routing with specialized memory roles and reduced decoding tokens over long generations. Code is available at https://github.com/Hunter-Wrynn/DLMR.

Related papers

• Homer: Understanding Long-form Videos with Hierarchical Memory and Agentic Reasoning [77.7531245219403]
  textscHomer is a Hierarchical Online Memory Exploration and Reasoning framework.<n>textscHomer's memory mirrors the multi-scale structure of long videos, ranging from raw perception, to recurring entities, to events connected by explicit temporal and causal relations.<n>textscHomer outperforms the previous best agent method by $+5.5$, $+10.8$, and $+4.4$ points on M3-Bench-robot, M3-Bench-web, and Video-MME-Long.
  arXiv  Detail & Related papers  (2026-07-01T05:53:09Z)
• MemLens: Benchmarking Multimodal Long-Term Memory in Large Vision-Language Models [50.25006399944962]
  Memory is essential for large vision-language models to handle long, multimodal interactions.<n>MEMLENS is a benchmark for memory in multimodal multi-session conversations.<n>We evaluate 27 LVLMs and 7 memory-augmented agents.
  arXiv  Detail & Related papers  (2026-05-14T14:41:17Z)
• From Verbatim to Gist: Distilling Pyramidal Multimodal Memory via Semantic Information Bottleneck for Long-Horizon Video Agents [78.30630000529133]
  We propose MM-Mem, a pyramidal multimodal memory architecture grounded in Fuzzy-Trace Theory.<n> MM-Mem memory structures hierarchically into a Sensory Buffer, Episodic Stream, and Symbolic.<n>Experiments confirm the effectiveness of MM-Mem on both offline and streaming tasks.
  arXiv  Detail & Related papers  (2026-03-02T05:12:45Z)
• WorldMM: Dynamic Multimodal Memory Agent for Long Video Reasoning [66.24870234484668]
  We introduce WorldMM, a novel multimodal memory agent that constructs and retrieves from multiple complementary memories.<n>WorldMM significantly outperforms existing baselines across five long video question-answering benchmarks.
  arXiv  Detail & Related papers  (2025-12-02T05:14:52Z)
• Agentic Learner with Grow-and-Refine Multimodal Semantic Memory [50.81667005063605]
  ViLoMem is a dual-stream memory framework that constructs compact, schema-based memory.<n>It encodes visual distraction patterns and logical reasoning errors, enabling MLLMs to learn from their successful and failed experiences.
  arXiv  Detail & Related papers  (2025-11-26T18:55:08Z)
• Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning [89.55738101744657]
  Large Language Models (LLMs) have demonstrated impressive capabilities across a wide range of NLP tasks, but they remain fundamentally stateless.<n>We present Memory-R1, a reinforcement learning framework that equips LLMs with the ability to actively manage and utilize external memory.
  arXiv  Detail & Related papers  (2025-08-27T12:26:55Z)
• Can Memory-Augmented Language Models Generalize on Reasoning-in-a-Haystack Tasks? [31.192647141427233]
  We propose MemReasoner, a new and simple memory-augmented LLM architecture.<n>MemReasoner is trained end-to-end, with optional supporting fact supervision.<n>We show strong generalization of MemReasoner on both single- and two-hop tasks.
  arXiv  Detail & Related papers  (2025-03-10T22:48:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.