EM^2Mem: Event-Centric Multimodal Memory for Large Language Models

• URL: http://arxiv.org/abs/2609.00551v1
• Date: Tue, 01 Sep 2026 01:38:41 GMT
• Title: EM^2Mem: Event-Centric Multimodal Memory for Large Language Models
• Abstract summary: We propose EM2Mem, an event-centric multimodal memory framework that aligns heterogeneous evidence to event anchors during memory construction.<n>Across three long-video QA benchmarks, EM2Mem improves average accuracy over the strongest memory baseline by 2.0, 2.4, and 3.7 points.
• Score: 52.3165591032625
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Multimodal memory offers a scalable interface for long-video question answering, but existing methods often retrieve captions, frames, transcripts, summaries, or graph facts as isolated fragments. Although searchable, such fragments are not generation-ready: language models must reconstruct cross-modal and temporal alignments at inference time, when context is limited and attribution is difficult. We propose EM^2Mem, an event-centric multimodal memory framework that binds heterogeneous evidence to event anchors during memory construction. Each event-indexed memory cell aligns multimodal records, temporal context, graph-linked relations, semantic facts, and provenance, enabling compact evidence readout over grounded multimodal events rather than modality-specific fragments. Across three long-video QA benchmarks, EM^2Mem improves average accuracy over the strongest memory baseline by 2.0, 2.4, and 3.7 points, improves strict event-level Top-5 evidence recall by 7.0 points, and reduces per-query latency by 4.67 times and total inference tokens by 63.66% (The code will be integrated into https://github.com/zjunlp/LightMem).

Related papers

• UniMem: Unifying Multimodal Memory and Control for Vision-Language-Action Models [8.70663346892131]
  We present UniMem, a framework that unifies high-level, multimodal memory and low-level control under one backbone.<n>We evaluate UniMem across five simulation and four hardware tasks targeting sequential and spatial memory.
  arXiv  Detail & Related papers  (2026-08-24T06:56:02Z)
• EverMemBench: Benchmarking Long-Term Interactive Memory in Large Language Models [16.865998112859604]
  We introduce EverMemBench, a benchmark featuring multi-party, multi-group conversations spanning over 1 million tokens.<n>EverMemBench evaluates memory systems across three dimensions through 1,000+ QA pairs.
  arXiv  Detail & Related papers  (2026-02-01T16:13:08Z)
• Beyond Dialogue Time: Temporal Semantic Memory for Personalized LLM Agents [68.84161689205779]
  Temporal Semantic Memory (TSM) is a memory framework that models semantic time for point-wise memory.<n>TSM consistently outperforms existing methods and achieves up to 12.2% absolute improvement in accuracy.
  arXiv  Detail & Related papers  (2026-01-12T12:24:44Z)
• Mem-Gallery: Benchmarking Multimodal Long-Term Conversational Memory for MLLM Agents [76.76004970226485]
  Long-term memory is a critical capability for multimodal large language model (MLLM) agents.<n>Mem-Gallery is a new benchmark for evaluating multimodal long-term conversational memory in MLLM agents.
  arXiv  Detail & Related papers  (2026-01-07T02:03:13Z)
• WorldMM: Dynamic Multimodal Memory Agent for Long Video Reasoning [66.24870234484668]
  We introduce WorldMM, a novel multimodal memory agent that constructs and retrieves from multiple complementary memories.<n>WorldMM significantly outperforms existing baselines across five long video question-answering benchmarks.
  arXiv  Detail & Related papers  (2025-12-02T05:14:52Z)
• LaMemo: Language Modeling with Look-Ahead Memory [50.6248714811912]
  We propose Look-Ahead Memory (LaMemo) that enhances the recurrence memory by incrementally attending to the right-side tokens. LaMemo embraces bi-directional attention and segment recurrence with an additional overhead only linearly proportional to the memory length. Experiments on widely used language modeling benchmarks demonstrate its superiority over the baselines equipped with different types of memory.
  arXiv  Detail & Related papers  (2022-04-15T06:11:25Z)
• Memory-Guided Semantic Learning Network for Temporal Sentence Grounding [55.31041933103645]
  We propose a memory-augmented network that learns and memorizes the rarely appeared content in TSG tasks. MGSL-Net consists of three main parts: a cross-modal inter-action module, a memory augmentation module, and a heterogeneous attention module.
  arXiv  Detail & Related papers  (2022-01-03T02:32:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.