Organize then Retrieve: Hierarchical Memory Navigation for Efficient Agents

• URL: http://arxiv.org/abs/2606.11680v1
• Date: Wed, 10 Jun 2026 05:49:14 GMT
• Title: Organize then Retrieve: Hierarchical Memory Navigation for Efficient Agents
• Abstract summary: HORMA is a hierarchical organize-and-Retrieve Memory Agent for large language models.<n>It decomposes working memory into two stages: structured memory construction and navigation-based retrieval.<n>It consistently achieves better efficiency-performance trade-offs and generalizes effectively to unseen tasks.
• Score: 29.28371047415946
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language model (LLM) agents struggle with long-horizon tasks due to their inherent statelessness, requiring all task-relevant information to be encoded in growing input contexts. The resulting degraded reasoning quality, increased inference cost, and higher latency necessitate efficient working memory mechanisms. However, existing approaches either rely on lossy compression or similarity-based retrieval, which often fail to capture temporal structure and causal dependencies required for multi-step agentic tasks. In this work, we present HORMA, a Hierarchical Organize-and-Retrieve Memory Agent that organizes experience into a file-system-like hierarchical structure, where summarized entities are linked to the corresponding raw trajectories, enabling efficient access without losing detailed information. HORMA decomposes working memory into two stages: structured memory construction and navigation-based retrieval. The construction module iteratively refines how experiences are structured by distinguishing between failures caused by missing information and those caused by misleading or overloaded context. The navigation module retrieves task-relevant context by traversing the hierarchy using a lightweight agent trained with reinforcement learning to select minimal yet sufficient context, thereby reducing latency along the critical execution path. Across ALFWorld, LoCoMo, and LongMemEval, HORMA improves task performance under constrained context budgets while requiring at most 22.17% of the baseline token usage in long conversation tasks. Compared to existing methods, it consistently achieves better efficiency-performance trade-offs and generalizes effectively to unseen tasks.

Related papers

• Knowledge Reutilization in Meta-Reinforcement Learning [51.07319647819108]
  We propose a meta-knowledge reutilization framework that learns task-level knowledge on a dynamics-simplified agent and transfers it to heterogeneous agents.<n>We show that our framework reduces final-step tracking error by 94.75% -- 99.79% compared with recent state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-06-16T16:32:28Z)
• Task-Aware Structured Memory for Dynamic Multi-modal In-Context Learning [40.577016426387694]
  Multi-modal large language models (MLLMs) depend on in-context learning (ICL) for rapid task adaptation.<n>Existing memory compression approaches rely on rigid token removal or sample-dependent importance estimation.<n>We introduce TASM, a training-free framework that addresses these limitations through task-aware, structure-preserving, and dynamically accessible memory construction.
  arXiv  Detail & Related papers  (2026-06-10T09:30:25Z)
• ContextWeaver: Selective and Dependency-Structured Memory Construction for LLM Agents [19.133532974091953]
  ContextWeaver is a selective and dependency-structured memory framework for large language model (LLM) agents.<n>It organizes an agent's interaction trace into a graph of reasoning steps and selects the relevant context for future actions.<n>It improves performance over a sliding-window baseline in pass@1, while reducing reasoning steps and token usage.
  arXiv  Detail & Related papers  (2026-04-24T23:39:49Z)
• AMA: Adaptive Memory via Multi-Agent Collaboration [54.490349689939166]
  We propose Adaptive Memory via Multi-Agent Collaboration (AMA), a novel framework that leverages coordinated agents to manage memory across multiple granularities.<n>AMA significantly outperforms state-of-the-art baselines while reducing token consumption by approximately 80% compared to full-context methods.
  arXiv  Detail & Related papers  (2026-01-28T08:09:49Z)
• SimpleMem: Efficient Lifelong Memory for LLM Agents [73.74399447715052]
  We introduce SimpleMem, an efficient memory framework based on semantic lossless compression.<n>We propose a three-stage pipeline designed to maximize information density and token utilization.<n> Experiments on benchmark datasets show that our method consistently outperforms baseline approaches in accuracy, retrieval efficiency, and inference cost.
  arXiv  Detail & Related papers  (2026-01-05T21:02:49Z)
• TALM: Dynamic Tree-Structured Multi-Agent Framework with Long-Term Memory for Scalable Code Generation [0.0]
  Agentic code generation requires large language models capable of complex context management and multi-step reasoning.<n>We propose TALM, a dynamic framework that integrates structured task decomposition, localized re-reasoning, and long-term memory mechanisms.<n> Experimental results on HumanEval, BigCodeBench, and ClassEval benchmarks demonstrate that TALM consistently delivers strong reasoning performance and high token efficiency.
  arXiv  Detail & Related papers  (2025-10-27T05:07:36Z)
• CAM: A Constructivist View of Agentic Memory for LLM-Based Reading Comprehension [55.29309306566238]
  Current Large Language Models (LLMs) are confronted with overwhelming information volume when comprehending long-form documents.<n>This challenge raises the imperative of a cohesive memory module, which can elevate vanilla LLMs into autonomous reading agents.<n>We draw inspiration from Jean Piaget's Constructivist Theory, illuminating three traits of the agentic memory -- structured schemata, flexible assimilation, and dynamic accommodation.
  arXiv  Detail & Related papers  (2025-10-07T02:16:30Z)
• MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents [84.62985963113245]
  We introduce MEM1, an end-to-end reinforcement learning framework that enables agents to operate with constant memory across long multi-turn tasks.<n>At each turn, MEM1 updates a compact shared internal state that jointly supports memory consolidation and reasoning.<n>We show that MEM1-7B improves performance by 3.5x while reducing memory usage by 3.7x compared to Qwen2.5-14B-Instruct on a 16-objective multi-hop QA task.
  arXiv  Detail & Related papers  (2025-06-18T19:44:46Z)
• Task Memory Engine (TME): Enhancing State Awareness for Multi-Step LLM Agent Tasks [0.0]
  We propose a lightweight and structured memory module that tracks task execution using a hierarchical Task Memory Tree (TMT)<n>TME is designed to be graph-aware, supporting reusable substeps, converging task paths, and shared dependencies.
  arXiv  Detail & Related papers  (2025-04-11T13:38:36Z)
• Fast Inference and Transfer of Compositional Task Structures for Few-shot Task Generalization [101.72755769194677]
  We formulate it as a few-shot reinforcement learning problem where a task is characterized by a subtask graph. Our multi-task subtask graph inferencer (MTSGI) first infers the common high-level task structure in terms of the subtask graph from the training tasks. Our experiment results on 2D grid-world and complex web navigation domains show that the proposed method can learn and leverage the common underlying structure of the tasks for faster adaptation to the unseen tasks.
  arXiv  Detail & Related papers  (2022-05-25T10:44:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.