HTM-EAR: Importance-Preserving Tiered Memory with Hybrid Routing under Saturation

• URL: http://arxiv.org/abs/2603.10032v1
• Date: Fri, 27 Feb 2026 15:35:04 GMT
• Title: HTM-EAR: Importance-Preserving Tiered Memory with Hybrid Routing under Saturation
• Abstract summary: HTM-EAR is a hierarchical tiered memory substrate that integrates HNSW-based working memory with archival storage.<n>When L1 reaches capacity, items are evicted using a weighted score of importance and usage.<n>We compare the full system against variants without cross-encoder re-ranking, without routing gates, with LRU eviction, and an oracle with unbounded memory.
• Score: 0.42303492200814446
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Memory constraints in long-running agents require structured management of accumulated facts while preserving essential information under bounded context limits. We introduce HTM-EAR, a hierarchical tiered memory substrate that integrates HNSW-based working memory (L1) with archival storage (L2), combining importance-aware eviction and hybrid routing. When L1 reaches capacity, items are evicted using a weighted score of importance and usage. Queries are first resolved in L1; if similarity or entity coverage is insufficient, retrieval falls back to L2, and candidates are re-ranked using a cross-encoder. We evaluate the system under sustained saturation (15,000 facts; L1 capacity 500; L2 capacity 5000) using synthetic streams across five random seeds and real BGL system logs. Ablation studies compare the full system against variants without cross-encoder re-ranking, without routing gates, with LRU eviction, and an oracle with unbounded memory. Under saturation, the full model preserves active-query precision (MRR = 1.000) while enabling controlled forgetting of stale history, approaching oracle active performance (0.997 +/- 0.003). In contrast, LRU minimizes latency (21.1 ms) but permanently evicts 2416 essential facts. On BGL logs, the full system achieves MRR 0.336, close to the oracle (0.370), while LRU drops to 0.069. Code is publicly available at: https://github.com/shubham-61291/HTM-EAR

Related papers

• LLMVisor: A Real-Time Latency Attribution Model for Multi-Tenant LLM Serving [12.697860184583483]
  Co-batching improves throughput but obscures per-tenant usage and limits control.<n>We present LLMVisor, a roofline-guided latency model that captures the memory-bound and compute-bound phases.<n>Compared to a token-count baseline, LLMVisor attains near-perfect R-squared and reduces relative error by up to 2.5x.
  arXiv  Detail & Related papers  (2026-08-09T00:28:05Z)
• Learning to Forget: Sleep-Inspired Memory Consolidation for Resolving Proactive Interference in Large Language Models [0.11080037957254413]
  Large language models (LLMs) suffer from proactive interference (PI): outdated information in the context window disrupts retrieval of current values.<n>We propose SleepGate, a framework that augments transformer-based LLMs with a learned sleep cycle over the key-value ( KV) cache.<n>SleepGate achieves 99.5% retrieval accuracy at PI depth 5 and 97.0% at depth 10, while all five baselines -- full KV cache, sliding window, H2O, StreamingLLM, and decay-only ablation -- remain below 18%.
  arXiv  Detail & Related papers  (2026-03-15T17:54:43Z)
• Resource-Efficient Iterative LLM-Based NAS with Feedback Memory [49.44875022114861]
  Neural Architecture Search (NAS) automates network design, but conventional methods demand substantial computational resources.<n>We propose a closed-loop pipeline leveraging large language models (LLMs) to iteratively generate, evaluate, and refine convolutional neural network architectures.
  arXiv  Detail & Related papers  (2026-03-12T16:00:22Z)
• The Missing Memory Hierarchy: Demand Paging for LLM Context Windows [0.0]
  We present Pichay, a demand paging system for context windows.<n>In live deployment over 681turns, the system reduces context consumption by up to 93% (5,038KB to 339KB).<n>Under extreme sustained pressure, the system remains operational but exhibits the expected thrashing pathology, with repeated fault-in of evicted content.
  arXiv  Detail & Related papers  (2026-03-09T23:38:32Z)
• AMV-L: Lifecycle-Managed Agent Memory for Tail-Latency Control in Long-Running LLM Systems [0.0]
  We present AMV-L, a memory-management framework that treats agent memory as a managed systems resource.<n>AMV-L improves throughput by 3.1x and reduces latency by 4.2x (median), 4.7x (p95), and 4.4x (p99), while reducing the fraction of requests exceeding 2s from 13.8% to 0.007%.
  arXiv  Detail & Related papers  (2026-02-22T00:11:20Z)
• Out of the Memory Barrier: A Highly Memory Efficient Training System for LLMs with Million-Token Contexts [68.79341332280062]
  Training Large Language Models (LLMs) on long contexts is severely constrained by prohibitive GPU memory overhead, not training time.<n>We introduce OOMB, a highly memory-efficient training system that directly confronts this barrier.<n>Our approach employs a chunk-recurrent training framework with on-the-fly activation recomputation, which maintains a constant activation memory footprint.
  arXiv  Detail & Related papers  (2026-02-02T13:52:40Z)
• Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Eviction [72.27673320976933]
  Diffusion Large Language Models (dLLMs) enable breakthroughs in reasoning and parallel decoding.<n>Current caching techniques accelerate decoding by storing full-layer states, yet impose substantial memory usage.<n>We propose Sparse-dLLM, the first training-free framework integrating dynamic cache eviction with sparse attention.
  arXiv  Detail & Related papers  (2025-08-04T16:14:03Z)
• ZSMerge: Zero-Shot KV Cache Compression for Memory-Efficient Long-Context LLMs [15.76582272387931]
  We propose ZSMerge, a dynamic KV cache compression framework for efficient cache management.<n>ZSMerge significantly enhances memory efficiency and inference speed with negligible performance degradation.
  arXiv  Detail & Related papers  (2025-03-13T03:36:03Z)
• Dynamic Low-Rank Sparse Adaptation for Large Language Models [54.1231638555233]
  Low-rank Sparse Adaptation (LoSA) is a novel method that seamlessly integrates low-rank adaptation into sparse LLM sparsity.<n>LoSA dynamically sparsifies the LoRA outcomes based on the corresponding sparse weights during fine-tuning.<n>LoSA can efficiently boost the efficacy of sparse LLMs within a few hours, without introducing any additional inferential burden.
  arXiv  Detail & Related papers  (2025-02-20T18:37:32Z)
• vTensor: Flexible Virtual Tensor Management for Efficient LLM Serving [53.972175896814505]
  Large Language Models (LLMs) are widely used across various domains, processing millions of daily requests. Large Language Models (LLMs) are widely used across various domains, processing millions of daily requests.
  arXiv  Detail & Related papers  (2024-07-22T14:37:58Z)
• Hierarchical Context Merging: Better Long Context Understanding for Pre-trained LLMs [61.40047491337793]
  We present Hierarchical cOntext MERging (HOMER), a new training-free scheme designed to overcome the limitations of large language models. HomeR uses a divide-and-conquer algorithm, dividing long inputs into manageable chunks. A token reduction technique precedes each merging, ensuring memory usage efficiency.
  arXiv  Detail & Related papers  (2024-04-16T06:34:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.