AgentIR: A Workload-Adaptive Cascade Retrieval Substrate for Long-Term Conversational Memory

• URL: http://arxiv.org/abs/2605.25092v1
• Date: Sun, 24 May 2026 14:14:13 GMT
• Title: AgentIR: A Workload-Adaptive Cascade Retrieval Substrate for Long-Term Conversational Memory
• Abstract summary: Lucene-class engines treat the index as static and the query as stateless, leaving the workload's structure unexploited.<n>A cascade router decides from the BM25 top-k margin alone and re-tunes across workloads without retraining.<n>Capacity on a shared 8-core VM rises from 154 to 1,400 concurrent agents (9x)
• Score: 1.8479558716666358
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Long-term conversational memory is a retrieval workload classical IR was not built for: the index grows during the query stream, query types shift intra-session, and the latency budget per retrieval is sub-10 ms. Lucene-class engines treat the index as static and the query as stateless, leaving the workload's structure unexploited. AgentIR treats fusion as a per-query decision along two axes: which fusion to apply (BM25, Dense, RRF, or agent-aware RRF), and whether the ~52 ms dense channel is worth running at all. The second axis is a confidence-triggered cascade router that decides from the BM25 top-k margin alone and re-tunes across workloads without retraining. On LongMemEval (n=500), where the dense channel does add information, the cascade skips 63% of queries at parity LLM-judged accuracy (2.67x faster under two judges, paired bootstrap p>=0.88); per-qtype thresholds extend this to 5.76x under 5-fold cross-validation. On LoCoMo (n=1,982), where BM25 alone is already the strongest single system, the same trigger auto-tunes to a 100% skip rate (132x faster, +0.089 Hit@5). Capacity on a shared 8-core VM rises from ~154 to ~1,400 concurrent agents (9x). Underneath the cascade, a time-partitioned index does O(log 1/epsilon) work independent of corpus size: 1234x corpus growth costs only 3.6x latency, ending in 1769x over sequential at sub-100 us p50 on 5M records. At parity quality with Lucene on 9 BEIR datasets up to 8.8M docs, the substrate runs 10x geo-mean over Pyserini 8T and 11x over PISA-1T BlockMax-WAND; an A100 reaches 1.8-39x over Pyserini 8T; chunked index build sustains 56.8K docs/sec on MS MARCO. Three subtle BM25/GPU correctness pitfalls that silently regress nDCG@10 by 6-8x are documented and fixed; post-fix CPU and GPU agree within 0.0002 nDCG@10 on all eight datasets that fit a single A100.

Related papers

• PARSER: Read in Parallel, Reason in Depth for Long-Context LLM Agents [85.82624962221026]
  We introduce PARSER, which decouples reading from reasoning.<n>A bank of lightweight subagents each bound to a single chunk read the entire document in parallel.<n>On multi-hop QA with contexts ranging from 7K to 896K tokens, PARSER with a 4B backbone outperforms the strongest sequential memory baseline.
  arXiv  Detail & Related papers  (2026-09-06T16:19:01Z)
• MediaWiki Code2Code Search: Neural Retrieval for the Semantic Discovery of Open-Source Software Entities [0.5279475826661643]
  We present Code2 InformationCode Search, a neural retrieval system for semantic code-to-code discovery.<n>By 1.29 million structural (functions, types, and templates) Code across 2,500+ large repositories, our system enables retrieval based on computational intent.<n>We employ a split-build architecture, decouplingintensive offline indexing from a CPU-only serving layer.
  arXiv  Detail & Related papers  (2026-07-29T11:07:07Z)
• Towards Retrieving Interaction Spaces for Agentic Search [62.666902610940525]
  Recent direct corpus (DCI) work shows that agents can interact with the raw corpus through shell tools such as 5.4 interaction and file reads.<n>We argue that the role of retrieval for agentic search is not just to select documents that fit in the LLM context window, but to construct an interaction space.
  arXiv  Detail & Related papers  (2026-06-05T03:47:40Z)
• Training-Free Lexical-Dense Fusion for Conversational-Memory Retrieval [0.0]
  Retrieving the few past turns that answer a new query is the retrieval bottleneck behind long-term conversational memory.<n>Recent work, Nano-Memory, shows that scoring a session by the maximum query-turn similarity beats mean-pooled session embeddings.<n>We replicate it and ask what a training-free, CPU-only retrieval stage should add around it.
  arXiv  Detail & Related papers  (2026-06-02T20:22:16Z)
• MISA: Mixture of Indexer Sparse Attention for Long-Context LLM Inference [75.41426145782751]
  We propose a drop-in replacement for the DSA indexer that treats its indexer heads as a pool of mixture-of-experts.<n> MISA matches the dense DSA indexer on LongBench across DeepSeek-V3.2 and GLM-5 while running with eight and four times fewer indexer heads respectively.<n>Our TileLang kernel delivers roughly a 3.82 times speedup over DSA's original indexer kernel on a single NVIDIA H200 GPU.
  arXiv  Detail & Related papers  (2026-05-08T07:19:34Z)
• More Is Not Always Better: Cross-Component Interference in LLM Agent Scaffolding [4.738949927143789]
  LLM agent systems are built by stacking components (planning, tools, memory, self-reflection, retrieval)<n>We run a full factorial experiment over all 25=32 subsets of five components on HotpotQA and GSM8K with Llama-3.1-8B/70B conditions.<n>The All-In system is consistently suboptimal: on HotpotQA, a single-tool agent surpasses All-In by 32% (F1 0.233 vs 0.177, p23)<n>Our findings suggest maximally-equipped agent should be replaced by subset selection via interaction-aware analysis.
  arXiv  Detail & Related papers  (2026-05-07T06:01:43Z)
• Delta-Based Neural Architecture Search: LLM Fine-Tuning via Code Diffs [48.83701310501069]
  Large language models (LLMs) show strong potential for neural architecture generation.<n>Existing approaches produce complete model implementations from scratch.<n>We propose Delta-Code Generation, where fine-tuned LLMs generate compact unified diffs.
  arXiv  Detail & Related papers  (2026-05-06T13:32:05Z)
• Benchmarking Complex Multimodal Document Processing Pipelines: A Unified Evaluation Framework for Enterprise AI [0.0]
  We built EnterpriseDocBench to take a swing at parsing fidelity, indexing efficiency, retrieval relevance, and generation groundedness.<n>We ran three pipelines through it -- BM25, dense embedding, and a hybrid -- all with the same GPT-5 generator.<n>One result that genuinely surprised us: factual accuracy on stated claims is 85.5%, but answer completeness averages 0.40.
  arXiv  Detail & Related papers  (2026-04-29T07:48:41Z)
• ZenBrain: A Neuroscience-Inspired 7-Layer Memory Architecture for Autonomous AI Systems [51.56484100374058]
  On LongMemEval-500, ZenBrain matches a long-context oracle's binary-judge accuracy to within 4.5 pp.<n> ZenBrain is a 7-layer neuroscience-inspired memory architecture.
  arXiv  Detail & Related papers  (2026-04-26T20:39:19Z)
• vstash: Local-First Hybrid Retrieval with Adaptive Fusion for LLM Agents [0.0]
  We present **vstash**, a local-first document memory system that combines vector similarity search with fulltext keyword matching.<n>All data resides in a single file using sqlite-vec for approximate nearest neighbor search and FTS5 for keyword matching.
  arXiv  Detail & Related papers  (2026-04-16T19:22:58Z)
• SelRoute: Query-Type-Aware Routing for Long-Term Conversational Memory Retrieval [0.0]
  We introduce SelRoute, a framework that routes each query to a specialized retrieval pipeline based on its query type.<n>On LongMemEval_M, SelRoute achieves Recall@5 of 0.800 with bge-base-en-v1.5 (109M parameters) and 0.786 with bge-small-en-v1.5 (109M parameters)<n>A zero-ML baseline using FTS5 alone achieves NDCG@5 of 0.692, already exceeding all published baselines on storage quality.
  arXiv  Detail & Related papers  (2026-04-02T18:02:59Z)
• Scaling DoRA: High-Rank Adaptation via Factored Norms and Fused Kernels [83.99688944263843]
  Weight-De Low-Rank Adaptation (DoRA) extends LoRA by decoupling weight magnitude from direction.<n>At d_in = 8192 and rank r = 384, a single module's norm requires about 512 MB of transient working memory in bf16.<n>A factored norm decomposes the squared norm into base, cross, and Gram terms computable through O(d_out r + r2) intermediates, eliminating the dense product.
  arXiv  Detail & Related papers  (2026-03-23T17:57:24Z)
• Chronicals: A High-Performance Framework for LLM Fine-Tuning with 3.51x Speedup over Unsloth [0.0]
  We present Chronicals, an open-source training framework achieving 3.5x speedup over Unsloth.<n>We provide complete mathematical foundations: online softmax correctness, FlashAttention IO complexity O(N2 d2 M-1), LoRA+ learning rate gradient approximations.
  arXiv  Detail & Related papers  (2026-01-06T00:00:55Z)
• DISTFLASHATTN: Distributed Memory-efficient Attention for Long-context LLMs Training [82.06732962485754]
  FlashAttention effectively reduces the quadratic peak memory usage to linear in training transformer-based large language models (LLMs) on a single GPU. We introduce DISTFLASHATTN, a memory-efficient attention mechanism optimized for long-context LLMs training. It achieves 1.67x and 1.26 - 1.88x speedup compared to recent Ring Attention and DeepSpeed-Ulysses.
  arXiv  Detail & Related papers  (2023-10-05T03:47:57Z)
• IRLI: Iterative Re-partitioning for Learning to Index [104.72641345738425]
  Methods have to trade between obtaining high accuracy while maintaining load balance and scalability in distributed settings. We propose a novel approach called IRLI, which iteratively partitions the items by learning the relevant buckets directly from the query-item relevance data. We mathematically show that IRLI retrieves the correct item with high probability under very natural assumptions and provides superior load balancing.
  arXiv  Detail & Related papers  (2021-03-17T23:13:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.