AWM: Answerable Working Memory for Long-Document VQA Agents

• URL: http://arxiv.org/abs/2608.25618v1
• Date: Wed, 26 Aug 2026 10:35:25 GMT
• Title: AWM: Answerable Working Memory for Long-Document VQA Agents
• Abstract summary: Working memory should carry answer-supporting evidence across page inspections for later grounded answering.<n>We introduce emphmemory-only answerability, a diagnostic that asks whether a reader can answer from the question and terminal working memory alone.<n>AWM-GRPO improves final-answer accuracy over the RAG baseline by 8.1 and 11.9 points on textscMMLongBench-Doc and textscLongDocURL and reduces the memory-missing-correct rate by 2.7 points over answer-only GRPO.
• Score: 57.741922050360074
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long-document visual question answering increasingly relies on VLM agents that retrieve candidate pages, inspect page images, write findings to working memory, and synthesize answers. Working memory should carry answer-supporting evidence across page inspections for later grounded answering, yet existing evaluation mainly checks final-answer correctness and evidence-page access. This creates a memory-quality blind spot: an agent may reach the right page and answer correctly while leaving behind memory too generic or incomplete to support answering once page context is removed. We introduce \emph{memory-only answerability}, a diagnostic that asks whether a reader can answer from the question and terminal working memory alone. Building on this diagnostic, \emph{Answerable Working Memory} (AWM) treats terminal working memory as an answerable evidence artifact, and AWM-GRPO incorporates this signal into the GRPO reward while preserving final-answer priority. Under GRPO, this reward assigns higher advantages to answer-correct trajectories whose terminal working memory remains answerable. On \textsc{MMLongBench-Doc}, even when gold evidence pages are provided, 42.5\% of correct answers still cannot be answered from terminal working memory alone. AWM-GRPO improves final-answer accuracy over the RAG baseline by 8.1 and 11.9 points on \textsc{MMLongBench-Doc} and \textsc{LongDocURL} and reduces the memory-missing-correct rate by 2.7 points over answer-only GRPO.

Related papers

• Entity-Memory Graph Retrieval Improves Evidence Coverage in Long-Conversation Question Answering [0.0]
  On 1,986 questions from ten LoCoMo conversations, graph retrieval raises official evidence recall at top-k 25 from 79.7468% to 84.4842%.<n>Four paper-eligible requested configurations support empirical robustness across the tested GPT-3.5 and DeepSeek extractors on both outcomes.
  arXiv  Detail & Related papers  (2026-08-28T05:03:33Z)
• MemOps: Benchmarking Lifecycle Memory Operations in Long-Horizon Conversations [50.24315431387575]
  We introduce MemOps, a benchmark that reformulates conversational memory as a sequence of lifecycle operations.<n>MemOps represents each memory event with a structured trace specifying its trigger, target, scope, state transition, and supporting evidence.<n>Across long-context, retrieval-based, parametric and managed-memory systems, MemOps disentangles failure modes that final-answer accuracy alone conceals.
  arXiv  Detail & Related papers  (2026-07-14T15:33:44Z)
• A-TMA: Decoupling State-Aware Memory Failures in Long-Term Agent Memory [7.7803190371583]
  We study emphghost memory, a state coordination failure in which old, current, and transition facts coexist in the memory bank, remain mixed during retrieval, and mislead the answer model.<n>We propose ATMA, a state aware overlay for existing memory systems. ATMA keeps superseded and transition records in the bank, and exposes current, historical, and transition labels to QA.
  arXiv  Detail & Related papers  (2026-07-02T09:28:29Z)
• MEMPROBE: Probing Long-Term Agent Memory via Hidden User-State Recovery [58.64329475683699]
  We argue that long-term memory should be evaluated as an auditable post-interaction artifact.<n>We instantiate this view in MEMPROBE, a benchmark in which a memory-equipped agent assists simulated users.<n>We find that successful assistance and recoverable memory behave as distinct capabilities.
  arXiv  Detail & Related papers  (2026-06-23T13:52:46Z)
• AdaMem: Learning What to Remember for Personalized Long-Horizon LLM Agents [79.03892269184145]
  We argue that what is worth remembering is role-dependent, and propose textbfAdaMem (Adaptive Memory)<n>AdaMem improves QA accuracy by up to textbf+9.0% over the uniform Mem0 baseline while shrinking memory volume by textbf9%.
  arXiv  Detail & Related papers  (2026-06-19T06:35:52Z)
• InfoMem: Training Long-Context Memory Agents with Answer-Conditioned Information Gain [26.098768400948614]
  Long-context tasks require LLMs to identify and preserve answer-relevant information from large contexts.<n>Existing RL-based chunk-wise agents either rely on sparse final-answer rewards or use lexical intermediate rewards for memory and retrieval actions.<n>We propose InfoMem, a reward mechanism for training chunk-wise memory agents that evaluates final-memory utility using answer-conditioned information.
  arXiv  Detail & Related papers  (2026-06-02T08:39:03Z)
• DeferMem: Query-Time Evidence Distillation via Reinforcement Learning for Long-Term Memory QA [3.56156695760535]
  Large language model (LLM) agents still struggle with long-term memory question answering.<n>We present DeferMem, a long-term memory framework that decouples this problem into high-recall candidate retrieval and query-conditioned evidence distillation.<n>On LoCoMo and LongMemEval-S, DeferMem surpasses strong baselines in QA accuracy and memory-system efficiency.
  arXiv  Detail & Related papers  (2026-05-21T12:36:46Z)
• SAGE: A Self-Evolving Agentic Graph-Memory Engine for Structure-Aware Associative Memory [40.68349164818526]
  We introduce SAGE, a Self-evolving Agentic Graph-memory Engine that models graph memory as a dynamic long-term memory substrate.<n> SAGE couples two roles: a memory writer that incrementally constucts structured graph memory from interaction histories, and a Graph Foundation Model-based memory reader to perform retrieval and provide feedback to the memory writer.
  arXiv  Detail & Related papers  (2026-05-12T12:47:43Z)
• AMA: Adaptive Memory via Multi-Agent Collaboration [54.490349689939166]
  We propose Adaptive Memory via Multi-Agent Collaboration (AMA), a novel framework that leverages coordinated agents to manage memory across multiple granularities.<n>AMA significantly outperforms state-of-the-art baselines while reducing token consumption by approximately 80% compared to full-context methods.
  arXiv  Detail & Related papers  (2026-01-28T08:09:49Z)
• Fine-Mem: Fine-Grained Feedback Alignment for Long-Horizon Memory Management [63.48041801851891]
  Fine-Mem is a unified framework designed for fine-grained feedback alignment.<n> Experiments on Memalpha and MemoryAgentBench demonstrate that Fine-Mem consistently outperforms strong baselines.
  arXiv  Detail & Related papers  (2026-01-13T11:06:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.