Beyond Retrieval: Analytic Memory for Multimodal Agents

• URL: http://arxiv.org/abs/2607.29440v2
• Date: Tue, 04 Aug 2026 06:43:57 GMT
• Title: Beyond Retrieval: Analytic Memory for Multimodal Agents
• Abstract summary: Long-term multimodal memory must support not only retrieving relevant information but also computing over observations accumulated across interactions.<n>We present AdaMM, a framework that jointly supports retrieval and analytic memory.<n>Experiments on two long-term multimodal memory benchmarks, MemEye and MemGallery, show that AdaMM improves performance by up to 11.3% and 6.9%, respectively.
• Score: 20.053051187012315
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Long-term multimodal memory must support not only retrieving relevant information but also computing over observations accumulated across interactions. Existing systems largely emphasize \emph{retrieval memory}, organizing interaction histories through summaries and indexes to return query-relevant information at multiple granularities, from high-level abstractions to underlying records. In this paper, we formulate \emph{analytic memory} as a complementary abstraction that organizes recurring multimodal observations into queryable structures supporting filtering, aggregation, ranking, and temporal comparison. We present AdaMM, a framework that jointly supports retrieval and analytic memory. Rather than relying on application-defined schemas, AdaMM extracts provenance-linked attribute-value observations from dialogue, images, and contextual metadata, discovers recurring field structures, and materializes them for analytical access. At inference time, a memory-aware planner decomposes queries into retrieval and analytic operations and routes each operation to the appropriate tools. Experiments on two long-term multimodal memory benchmarks, MemEye and MemGallery, show that AdaMM improves performance by up to 11.3\% and 6.9\%, respectively.

Related papers

• GraphMemix: Query-Aware Evidence Forests for Long-Term Multimodal Agent Memory [87.58855368314693]
  We propose GraphMemix, a graph memory framework that models memory organization as query-aware evidence-forest construction.<n> Experimental results across four long-term multimodal memory benchmarks demonstrate significant improvements with different foundation models.
  arXiv  Detail & Related papers  (2026-08-27T11:28:49Z)
• Infini Memory: Maintainable Topic Documents for Long-Term LLM Agent Memory [19.802226925602927]
  Long-term LLM agents need persistent memory that can track changing facts and provide relevant evidence across sessions.<n>We propose Infini Memory, a maintainable text-based persistent memory architecture that treats agent memory as topic-structured documents.
  arXiv  Detail & Related papers  (2026-06-09T10:31:51Z)
• MemSearch-o1: Empowering Large Language Models with Reasoning-Aligned Memory Growth in Agentic Search [59.98962867649009]
  We propose MemSearch-o1, an agentic search framework built on reasoning-aligned memory growth and retracing.<n>MemSearch-o1 grows fine-grained memory fragments from memory seed tokens from queries, then retraces and deeply refines the memory via a contribution function, and finally reorganizes a globally connected memory path.
  arXiv  Detail & Related papers  (2026-04-19T05:35:06Z)
• TA-Mem: Tool-Augmented Autonomous Memory Retrieval for LLM in Long-Term Conversational QA [8.036549927091286]
  Large Language Model (LLM) has exhibited strong reasoning ability in text-based contexts across various domains.<n>The limitation of context window poses challenges for the model on long-range inference tasks.<n>We introduce a tool-augmented autonomous memory retrieval framework (TA-Mem)<n>The TA-Mem is evaluated on the LoCoMo dataset, achieving significant performance improvements over existing baseline approaches.
  arXiv  Detail & Related papers  (2026-03-10T07:27:01Z)
• Beyond RAG for Agent Memory: Retrieval by Decoupling and Aggregation [22.803751188961865]
  We argue retrieval should move beyond similarity matching and instead operate over latent components.<n>We propose xMemory, which builds a hierarchy of intact units and maintains a searchable high-level node organisation.
  arXiv  Detail & Related papers  (2026-02-02T12:04:58Z)
• AMA: Adaptive Memory via Multi-Agent Collaboration [54.490349689939166]
  We propose Adaptive Memory via Multi-Agent Collaboration (AMA), a novel framework that leverages coordinated agents to manage memory across multiple granularities.<n>AMA significantly outperforms state-of-the-art baselines while reducing token consumption by approximately 80% compared to full-context methods.
  arXiv  Detail & Related papers  (2026-01-28T08:09:49Z)
• Mem-Gallery: Benchmarking Multimodal Long-Term Conversational Memory for MLLM Agents [76.76004970226485]
  Long-term memory is a critical capability for multimodal large language model (MLLM) agents.<n>Mem-Gallery is a new benchmark for evaluating multimodal long-term conversational memory in MLLM agents.
  arXiv  Detail & Related papers  (2026-01-07T02:03:13Z)
• WorldMM: Dynamic Multimodal Memory Agent for Long Video Reasoning [66.24870234484668]
  We introduce WorldMM, a novel multimodal memory agent that constructs and retrieves from multiple complementary memories.<n>WorldMM significantly outperforms existing baselines across five long video question-answering benchmarks.
  arXiv  Detail & Related papers  (2025-12-02T05:14:52Z)
• From Single to Multi-Granularity: Toward Long-Term Memory Association and Selection of Conversational Agents [79.87304940020256]
  Large Language Models (LLMs) have been widely adopted in conversational agents.<n>MemGAS is a framework that enhances memory consolidation by constructing multi-granularity association, adaptive selection, and retrieval.<n> Experiments on four long-term memory benchmarks demonstrate that MemGAS outperforms state-of-the-art methods on both question answer and retrieval tasks.
  arXiv  Detail & Related papers  (2025-05-26T06:13:07Z)
• Rethinking Memory in AI: Taxonomy, Operations, Topics, and Future Directions [55.19217798774033]
  Memory is a fundamental component of AI systems, underpinning large language models (LLMs)-based agents.<n>In this survey, we first categorize memory representations into parametric and contextual forms.<n>We then introduce six fundamental memory operations: Consolidation, Updating, Indexing, Forgetting, Retrieval, and Compression.
  arXiv  Detail & Related papers  (2025-05-01T17:31:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.