Dual-Grained Agent Memory and Shapley Context Attribution for Multimodal Agentic Learner

• URL: http://arxiv.org/abs/2608.23268v1
• Date: Mon, 24 Aug 2026 13:56:01 GMT
• Title: Dual-Grained Agent Memory and Shapley Context Attribution for Multimodal Agentic Learner
• Abstract summary: multimodal large language models (MLLMs) deliver impressive perception yet still falter on scientific and mathematical reasoning.<n>We propose DG-Mem, a dual-grained agentic memory framework that augments a frozen MLLM with a non- externally stored memory built once from training-time rollouts and consulted read-only at test time.
• Score: 55.80248281761638
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Frontier multimodal large language models (MLLMs) deliver impressive perception yet still falter on scientific and mathematical reasoning. Parameter-level adaptation is unavailable for closed-weight or on-device backbones, and stateless prompting forfeits any compounding benefit from problems already solved. We propose \textbf{DG-Mem}, a dual-grained agentic memory framework that augments a frozen MLLM with a non-parametric, externally stored memory built once from training-time rollouts and consulted read-only at test time. Motivated by the Complementary Learning Systems (CLS) account of human memory, DG-Mem factors its store into an instance-grounded exemplar memory and a category-level schema memory of IF-THEN rules, with a transient reflection store mediating their construction so that schemas are synthesized only from abstract reflections, never from exemplar text. Two design choices distinguish DG-Mem: an online concept categorizer that grows the category space incrementally during training rather than committing to a predefined taxonomy, and a Shapley context attribution procedure that decomposes correctness across the entire retrieved rule set and yields a per-rule utility that re-weights retrieval at test time. The pipeline introduces no gradient updates and is deployable on closed-weight or on-device backbones. Across MathVista, MMMU, and MMMU-Pro on four open-weight and proprietary backbones (Qwen3.5-27B, Qwen3.5-122B-A10B, GPT-5-Nano, Gemini-3-Flash), DG-Mem improves consistently over no-memory and competitive memory baselines.

Related papers

• Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents [72.3752981234916]
  Large Language Model (LLM) agents increasingly rely on external memory systems to accumulate experience across tasks.<n>We present Memory as a Controlled Process (MemCon), a framework that models memory operations as a Markov Decision Process.<n>MemCon consistently outperforms multiple memory baselines by up to 15.2 points in task success while reducing token consumption by 5-20%.
  arXiv  Detail & Related papers  (2026-07-15T08:32:40Z)
• Rethinking Memory as Continuously Evolving Connectivity [63.58181883607843]
  Existing memory-augmented LLM agents treat memory as a static repository with pre-defined representations and fixed retrieval pipelines.<n>We propose FluxMem, a connectivity-evolving memory framework that models memory as a heterogeneous graph and progressively refines its topology through three stages.<n>Across three fundamentally distinct benchmarks including LoCoMo, Mind2Web, and GAIA, FluxMem achieves consistent state-of-the-art performance, demonstrating strong adaptation and generalization in complex agentic environments.
  arXiv  Detail & Related papers  (2026-05-27T17:35:34Z)
• MemGym: a Long-Horizon Memory Environment for LLM Agents [69.79226770543049]
  We present MemGym, a benchmark for agentic memory that unifies existing agent gyms and in-house memory-grounded pipelines.<n>MemGym reports memory-isolated scores that decouple memory performance from reasoning, retrieval, and tool-use ability.<n>Our synthetic pipelines for MEMGYM-CODEQA and MEMGYM-DR are length-controllable, ablation-verified at every stage, and tightly aligned with downstream scenarios.
  arXiv  Detail & Related papers  (2026-05-20T07:25:33Z)
• CALMem : Application-Layer Dual Memory for Conversational AI [0.0]
  CALMem is an application-layer dual memory architecture that gives conversational assistants virtually effective context without any modification to the underlying model.<n>A token-budget-adaptive injection mechanism, called the MOIM, automatically retrieves and injects relevant past context each turn.<n>We describe the architecture, design decisions, and performance characteristics, and analyse the trade-offs that guided each implementation choice.
  arXiv  Detail & Related papers  (2026-05-20T05:23:57Z)
• Memory as Metabolism: A Design for Companion Knowledge Systems [0.0]
  This paper proposes a companion-specific governance profile for single-user LLM memory.<n>The design principle is that personal LLM memory is a companion system.<n>The safety story at the single-agent level is partial, and the paper is explicit about what it does and does not solve.
  arXiv  Detail & Related papers  (2026-04-13T20:22:53Z)
• MemMA: Coordinating the Memory Cycle through Multi-Agent Reasoning and In-Situ Self-Evolution [52.29303869050117]
  Memory-augmented LLM agents maintain external memory banks to support long-horizon interaction.<n>MemMA is a plug-and-play multi-agent framework that coordinates the memory cycle along both the forward and backward paths.
  arXiv  Detail & Related papers  (2026-03-19T10:15:59Z)
• D-Mem: A Dual-Process Memory System for LLM Agents [3.5426740232689604]
  We introduce D-Mem, a dual-process memory system.<n>It retains lightweight vector retrieval for routine queries while establishing an exhaustive Full Deliberation module as a high-fidelity fallback.<n>Experiments on the LoCoMo and RealTalk benchmarks using GPT-4o-mini and Qwen3-235B-Instruct demonstrate the efficacy of our approach.
  arXiv  Detail & Related papers  (2026-03-19T08:55:22Z)
• TA-Mem: Tool-Augmented Autonomous Memory Retrieval for LLM in Long-Term Conversational QA [8.036549927091286]
  Large Language Model (LLM) has exhibited strong reasoning ability in text-based contexts across various domains.<n>The limitation of context window poses challenges for the model on long-range inference tasks.<n>We introduce a tool-augmented autonomous memory retrieval framework (TA-Mem)<n>The TA-Mem is evaluated on the LoCoMo dataset, achieving significant performance improvements over existing baseline approaches.
  arXiv  Detail & Related papers  (2026-03-10T07:27:01Z)
• Neuromem: A Granular Decomposition of the Streaming Lifecycle in External Memory for LLMs [31.958776309037546]
  We present Neuromem, a scalable testbed that benchmarks External Memory Modules under an interleaved insertion-and-retrieval protocol.<n>We observe that performance typically degrades as memory grows across rounds, and time-related queries remain the most challenging category.
  arXiv  Detail & Related papers  (2026-02-15T02:53:37Z)
• LatentMem: Customizing Latent Memory for Multi-Agent Systems [44.59989123744384]
  We propose LatentMem, a learnable multi-agent memory framework designed to customize agent-specific memories in a token-efficient manner.<n>Specifically, LatentMem comprises an experience bank that stores raw interaction trajectories in a lightweight form, and a memory composer that synthesizes compact latent memories conditioned on retrieved experience and agent-specific contexts.
  arXiv  Detail & Related papers  (2026-02-03T03:03:16Z)
• Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning [89.55738101744657]
  Large Language Models (LLMs) have demonstrated impressive capabilities across a wide range of NLP tasks, but they remain fundamentally stateless.<n>We present Memory-R1, a reinforcement learning framework that equips LLMs with the ability to actively manage and utilize external memory.
  arXiv  Detail & Related papers  (2025-08-27T12:26:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.