Cooperative Memory Paging with Keyword Bookmarks for Long-Horizon LLM Conversations

• URL: http://arxiv.org/abs/2604.12376v1
• Date: Tue, 14 Apr 2026 07:06:35 GMT
• Title: Cooperative Memory Paging with Keyword Bookmarks for Long-Horizon LLM Conversations
• Abstract summary: We propose cooperative paging: segments are replaced with minimal keyword bookmarks, and the model is given a recall() tool to retrieve full content on demand.<n>On the LoCoMo benchmark, cooperative paging achieves the highest answer quality among six methods -- outperforming truncation, BM25, word-overlap retrieval, a search-tool baseline, and full context.<n>We then study the paging design space with a 5x4 ablation over boundary strategies and eviction policies (3,176 synthetic probes, 1,600 LoCoMo probes)
• Score: 2.6382975801439836
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: When LLM conversations grow beyond the context window, old content must be evicted -- but how does the model recover it when needed? We propose cooperative paging: evicted segments are replaced with minimal keyword bookmarks ([pN:keywords], ~8-24 tokens each), and the model is given a recall() tool to retrieve full content on demand. On the LoCoMo benchmark (10 real multi-session conversations, 300+ turns), cooperative paging achieves the highest answer quality among six methods -- outperforming truncation, BM25, word-overlap retrieval, a search-tool baseline, and full context -- on four models (GPT-4o-mini, DeepSeek-v3.2, Claude Haiku, GLM-5), confirmed by four independent LLM judges ($p=0.017$, paired bootstrap). We then study the paging design space with a 5x4 ablation over boundary strategies and eviction policies (3,176 synthetic probes, 1,600 LoCoMo probes). Key findings: (1) coarse fixed-size pages (fixed_20) reach 96.7% while content-aware topic_shift collapses to 56.7%; (2) eviction policy choice is data-dependent (FIFO best on synthetic, LFU on LoCoMo); (3) two bookmark generation strategies improve over the heuristic baseline (+4.4 and +8.7 E2E points); (4) the remaining bottleneck is bookmark discrimination -- the model triggers recall() 96% of the time but selects the correct page only 57% when bookmarks are insufficiently distinctive. Keyword specificity alone accounts for a 25 percentage point accuracy difference.

Related papers

• ClinTraceBench: Source-Verifiable Longitudinal Clinical Reasoning over EHR-Derived Dialogues [11.42809060379683]
  We introduce ClinTraceBench: 385 MIMIC-IV-derived verified dialogues with event-ID provenance, a nine-task taxonomy, and L0--L4 deterministic + L5 human-audit validation.<n>We evaluate eight history representation strategies---a no-context floor, textitlast-visit-only, textitfull-context, BGE-M3 textitdense-retrieval, two compression schemes, and two agentic-memory systems.
  arXiv  Detail & Related papers  (2026-09-01T11:51:37Z)
• Trust Before Fusion: QIMG-7 and Source-Aware Resolution for Polluted Multimodal RAG [45.88028371034407]
  Multimodal retrieval-augmented generation (RAG) is often evaluated with clean evidence, yet real retrieval can return topically relevant but unreliable content.<n>We introduce QIMG-7, a benchmark for multimodal retrieval pollution in multi-sentence factual QA.<n>We propose source-aware trust resolution (SATR), a training-free approach that compares Parametric, Text-only, and Full-MM candidate answers.
  arXiv  Detail & Related papers  (2026-07-12T15:13:09Z)
• TokenMizer: Graph-Structured Session Memory for Long-Horizon LLM Context Management [0.0]
  We present TokenMizer, an open-source proxy system that models large language model sessions as a typed knowledge graph.<n>It produces resume blocks averaging 78 tokens (range: 42-124) - 2x smaller than evaluated baselines.<n>Across all sessions, TokenMizer mean task recall 51.0%, decision recall 46.6%, and file recall 58.7%.
  arXiv  Detail & Related papers  (2026-06-04T16:12:28Z)
• MEME: Multi-entity & Evolving Memory Evaluation [76.57263966646404]
  MEME defines six tasks spanning the full space defined by the multi-entity and evolving axes.<n> evaluating six memory systems spanning three memory paradigms on 100 controlled episodes.<n>We find that all systems collapse on dependency reasoning under the default configuration.
  arXiv  Detail & Related papers  (2026-05-12T17:55:10Z)
• MemReranker: Reasoning-Aware Reranking for Agent Memory Retrieval [37.54115468116941]
  This report introduces MemReranker, a reranking model family (0.6B/4B) built on Qwen3-Reranker through multi-stage LLM knowledge distillation.<n>On the memory retrieval benchmark, MemReranker-0.6B substantially outperforms BGE-Reranker and matches open-source 4B/8B models as well as GPT-4o-mini on key metrics.<n>MemReranker-4B further achieves 0.737 MAP, with several metrics on par with Gemini-3-Flash, while maintaining inference latency at only 10-20% of large models.
  arXiv  Detail & Related papers  (2026-05-07T12:33:57Z)
• DualView: Adaptive Local-Global Fusion for Multi-Hop Document Reranking [20.174286204915756]
  Multi-hop question answering requires aggregating information from multiple documents.<n>We present an efficient dual-view cascaded reranking framework for multi-hop document reranking.
  arXiv  Detail & Related papers  (2026-04-13T08:56:09Z)
• SmartSearch: How Ranking Beats Structure for Conversational Memory Retrieval [0.0]
  SmartSearch retrieves from raw, unstructured conversation history using a fully deterministic pipeline.<n>SmartSearch achieves 93.5% on LoCoMo and 88.4% on LongMemEval-S benchmarks.
  arXiv  Detail & Related papers  (2026-03-16T17:53:21Z)
• The Missing Memory Hierarchy: Demand Paging for LLM Context Windows [0.0]
  We present Pichay, a demand paging system for context windows.<n>In live deployment over 681turns, the system reduces context consumption by up to 93% (5,038KB to 339KB).<n>Under extreme sustained pressure, the system remains operational but exhibits the expected thrashing pathology, with repeated fault-in of evicted content.
  arXiv  Detail & Related papers  (2026-03-09T23:38:32Z)
• MultiHaystack: Benchmarking Multimodal Retrieval and Reasoning over 40K Images, Videos, and Documents [57.32877731797049]
  MultiHaystack is the first benchmark designed to evaluate both retrieval and reasoning under large-scale, cross-modal conditions.<n>We find that models perform competitively when provided with the corresponding evidence, but their performance drops sharply when required to retrieve that evidence from the full corpus.
  arXiv  Detail & Related papers  (2026-03-05T21:43:02Z)
• Domain-Adaptive and Scalable Dense Retrieval for Content-Based Recommendation [0.0]
  We present a scalable dense retrieval system based on a two-tower bi-encoder, fine-tuned on the Amazon Reviews 2023 (Fashion) subset.<n>We construct training pairs from review text (as a query proxy) and item metadata (as the positive document) and fine-tune on 50,000 sampled interactions with a maximum sequence length of 500 tokens.<n>On a review-to-title benchmark over 826,402 catalog items, our approach improves Recall@10 from 0.26 (BM25) to 0.66.
  arXiv  Detail & Related papers  (2026-01-31T20:58:23Z)
• Assessing LLM Reliability on Temporally Recent Open-Domain Questions [15.456770184839726]
  Large Language Models (LLMs) are increasingly deployed for open-domain question answering.<n>We investigate how four open-source LLMs respond to 15,000 recent Reddit questions.<n>All models achieve over 99% cosine similarity with references despite less than 8% BLEU-1 overlap.
  arXiv  Detail & Related papers  (2026-01-17T21:33:27Z)
• HuggingR$^{4}$: A Progressive Reasoning Framework for Discovering Optimal Model Companions [50.61510609116118]
  HuggingR$4$ is a novel framework that combines Reasoning, Retrieval, Refinement, and Reflection to efficiently select models.<n>It attains a workability rate of 92.03% and a reasonability rate of 82.46%, surpassing existing method by 26.51% and 33.25% respectively.
  arXiv  Detail & Related papers  (2025-11-24T03:13:45Z)
• CIMemories: A Compositional Benchmark for Contextual Integrity of Persistent Memory in LLMs [62.116710797795314]
  Large Language Models (LLMs) increasingly use persistent memory from past interactions to enhance personalization and task performance.<n>We present CIMemories, a benchmark for evaluating whether LLMs appropriately control information flow from memory based on task context.
  arXiv  Detail & Related papers  (2025-11-18T21:51:23Z)
• Document Attribution: Examining Citation Relationships using Large Language Models [62.46146670035751]
  We propose a zero-shot approach that frames attribution as a straightforward textual entailment task.<n>We also explore the role of the attention mechanism in enhancing the attribution process.
  arXiv  Detail & Related papers  (2025-05-09T04:40:11Z)
• Benchmarking Reasoning Robustness in Large Language Models [76.79744000300363]
  We find significant performance degradation on novel or incomplete data.<n>These findings highlight the reliance on recall over rigorous logical inference.<n>This paper introduces a novel benchmark, termed as Math-RoB, that exploits hallucinations triggered by missing information to expose reasoning gaps.
  arXiv  Detail & Related papers  (2025-03-06T15:36:06Z)
• NoLiMa: Long-Context Evaluation Beyond Literal Matching [100.00398424275501]
  NoLiMa is a benchmark extending the needle-in-a-haystack (NIAH) test.<n>It requires models to infer latent associations to locate the needle within the haystack.<n>We evaluate 13 popular large language models that claim to support contexts of at least 128K tokens.
  arXiv  Detail & Related papers  (2025-02-07T18:49:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.