HMARS: A Hierarchical Multi-Agent Memory System for Long-Context Reasoning

• URL: http://arxiv.org/abs/2606.28349v1
• Date: Wed, 03 Jun 2026 07:15:11 GMT
• Title: HMARS: A Hierarchical Multi-Agent Memory System for Long-Context Reasoning
• Abstract summary: Long-context reasoning requires models to access, retrieve, and integrate evidence scattered across documents, dialogues, and interaction histories.<n>We propose HMARS, a hierarchical multi-agent memory system that treats long contexts as managed memory rather than a flat retrieval corpus.
• Score: 17.783618926697066
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Long-context reasoning requires models to access, retrieve, and integrate evidence scattered across documents, dialogues, and accumulated interaction histories. Standard retrieval-augmented generation reduces this problem to top-$K$ chunk retrieval, but such passive access can discard relevant evidence before reasoning begins, especially when relevance depends on broader context. We propose HMARS, a hierarchical multi-agent memory system that treats long contexts as managed memory rather than a flat retrieval corpus. Sub-agents maintain grounded access to bounded memory regions, mid-agents manage regional context and provide query-specific coordination, and a frontier model performs final reasoning over retrieved evidence pages. To evaluate this view, we construct two diagnostic benchmarks targeting evidence breadth and context-dependent relevance. Across long-document and multi-turn memory tasks, HMARS achieves the best overall performance against retrieval, reranking, full-context, graph-based, and agentic long-context baselines. Evidence coverage analysis further shows that its gains come from retrieving the required supporting evidence more completely, rather than merely changing the final answer prompt.

Related papers

• VisDocAgentBench: Benchmarking Agents for Visually Rich Document Retrieval [65.89247522243959]
  VisDocAgentBench is a closed-corpus benchmark comparing static and agentic retrieval under a ranked-output contract.<n>It contains 2,375 pages from 100 documents and 120 unique-target queries balanced across direct, one-bridge, and two-bridge evidence structures.<n>A strong late-interaction visual retriever reaches 97.50% Recall@1 on direct items but 2.50% on two-bridge items, exposing the limits of query--target matching when relevance depends on corpus context.
  arXiv  Detail & Related papers  (2026-08-18T15:23:06Z)
• CMT-RAG: Complementary Memory Traces for Multi-turn Multi-hop RAG [17.889279174355227]
  We introduce MuMu-QA, a benchmark for multi-turn multi-hop RAG with explicit cross-turn sub-question dependency annotations, and CMT-RAG, a complementary memory framework for this setting.<n>MuMu-QA and corpus-level RAG benchmarks show that CMT-RAG consistently outperforms five categories of RAG baselines in answer accuracy.
  arXiv  Detail & Related papers  (2026-07-29T04:50:41Z)
• Hierarchical Evidence-Driven Reasoning for Long Document Understanding [95.51688464037096]
  We introduce HIEVI-RAG, a hierarchical, evidence-driven multimodal RAG framework for closed-domain document understanding.<n>Our framework significantly outperforms existing open-source baselines and exceeds the strongest reported baseline by an average of 8.05% in accuracy.
  arXiv  Detail & Related papers  (2026-07-06T03:08:28Z)
• ReContext: Recursive Evidence Replay as LLM Harness for Long-Context Reasoning [78.94047086588078]
  We propose Recursive Evidence as LLM Harness for Long-Context Reasoning.<n>It is a training-free inference method for improving long-context reasoning.<n> Experiments on eight long-context datasets with 128K context length show it consistently improves evidence utilization.
  arXiv  Detail & Related papers  (2026-07-02T17:59:26Z)
• MAGE-RAG: Multigranular Adaptive Graph Evidence for Agentic Multimodal RAG in Long-Document QA [14.865891513918278]
  Long-document multimodal question answering requires a system to locate sparse evidence in long PDFs.<n>This paper proposes MAGE-RAG, a multigranular adaptive graph evidence framework for long-document multimodal QA.
  arXiv  Detail & Related papers  (2026-06-14T16:34:15Z)
• Trace Only What You Need: Structure-Aware On-Demand Hypergraph Memory for Long-Document Question Answering [11.17007249581283]
  Long-document question answering requires large language models to reason over evidence scattered across lengthy documents.<n>Existing structured RAG methods face three limitations: costly query-agnostic knowledge organization, insufficient use of original document structure, and no reuse of historical reasoning experience.<n>We propose DocTrace, a multi-agent RAG framework for long-document QA that supports query-triggered knowledge organization, document-structure-aware and experience-guided reasoning.
  arXiv  Detail & Related papers  (2026-06-09T14:29:06Z)
• MARDoc: A Memory-Aware Refinement Agent Framework for Multimodal Long Document QA [23.25080415779712]
  We propose a Memory-Aware Refinement Agent framework that decouples long-document QA into three specialized agents.<n>Across iterations, the agents rely on a dynamically updated structured memory rather than a full accumulated interaction history.<n>Experiments on MMLongBench-Doc and DocBench show that MARDoc achieves strong results, outperforming same-backbone baselines.
  arXiv  Detail & Related papers  (2026-06-04T06:23:01Z)
• VLD-RAG: Agentic Vision-Language Retrieval-Augmented Generation for Long, Visually-Rich Multi-Page Documents [0.0]
  We present VLD-RAG, an agentic multimodal RAG framework for multi-page evidence retrieval and cross-page reasoning over long documents.<n>VLD-RAG builds a page-preserving multimodal index that stores parsed text, page-level metadata, and dense visual representations.<n>A verifier-guided agent workflow coordinates a Retrieval Agent, Answer Agent, and Validation Agent to broaden evidence coverage, detect missing citations, and refine retrieval requests.
  arXiv  Detail & Related papers  (2026-05-21T13:33:38Z)
• NaviRAG: Towards Active Knowledge Navigation for Retrieval-Augmented Generation [50.16741209529908]
  NaviRAG is a novel framework that shifts from passive segment retrieval to active knowledge navigation.<n>We show that NaviRAG consistently improves both retrieval recall and end-to-end answer performance over conventional RAG baselines.
  arXiv  Detail & Related papers  (2026-04-14T14:07:01Z)
• AMA: Adaptive Memory via Multi-Agent Collaboration [54.490349689939166]
  We propose Adaptive Memory via Multi-Agent Collaboration (AMA), a novel framework that leverages coordinated agents to manage memory across multiple granularities.<n>AMA significantly outperforms state-of-the-art baselines while reducing token consumption by approximately 80% compared to full-context methods.
  arXiv  Detail & Related papers  (2026-01-28T08:09:49Z)
• Resolving Evidence Sparsity: Agentic Context Engineering for Long-Document Understanding [49.26132236798123]
  Vision Language Models (VLMs) have gradually become a primary approach in document understanding.<n>We propose SLEUTH, a multi agent framework that orchestrates a retriever and four collaborative agents in a coarse to fine process.<n>The framework identifies key textual and visual clues within the retrieved pages, filters for salient visual evidence such as tables and charts, and analyzes the query to devise a reasoning strategy.
  arXiv  Detail & Related papers  (2025-11-28T03:09:40Z)
• GERE: Generative Evidence Retrieval for Fact Verification [57.78768817972026]
  We propose GERE, the first system that retrieves evidences in a generative fashion. The experimental results on the FEVER dataset show that GERE achieves significant improvements over the state-of-the-art baselines.
  arXiv  Detail & Related papers  (2022-04-12T03:49:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.