Structured Distillation for Personalized Agent Memory: 11x Token Reduction with Retrieval Preservation

• URL: http://arxiv.org/abs/2603.13017v1
• Date: Fri, 13 Mar 2026 14:21:58 GMT
• Title: Structured Distillation for Personalized Agent Memory: 11x Token Reduction with Retrieval Preservation
• Abstract summary: Long conversations with an AI agent create a simple problem for one user: the history is useful, but carrying it verbatim is expensive.<n>We study personalized agent memory: one user's conversation history with an agent, distilled into a compact retrieval layer for later search.<n> Applied to 4,182 conversations (14,340 exchanges) from 6 software engineering projects, the method reduces average exchange length from 371 to 38 tokens, yielding 11x compression.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Long conversations with an AI agent create a simple problem for one user: the history is useful, but carrying it verbatim is expensive. We study personalized agent memory: one user's conversation history with an agent, distilled into a compact retrieval layer for later search. Each exchange is compressed into a compound object with four fields (exchange_core, specific_context, thematic room_assignments, and regex-extracted files_touched). The searchable distilled text averages 38 tokens per exchange. Applied to 4,182 conversations (14,340 exchanges) from 6 software engineering projects, the method reduces average exchange length from 371 to 38 tokens, yielding 11x compression. We evaluate whether personalized recall survives that compression using 201 recall-oriented queries, 107 configurations spanning 5 pure and 5 cross-layer search modes, and 5 LLM graders (214,519 consensus-graded query-result pairs). The best pure distilled configuration reaches 96% of the best verbatim MRR (0.717 vs 0.745). Results are mechanism-dependent. All 20 vector search configurations remain non-significant after Bonferroni correction, while all 20 BM25 configurations degrade significantly (effect sizes |d|=0.031-0.756). The best cross-layer setup slightly exceeds the best pure verbatim baseline (MRR 0.759). Structured distillation compresses single-user agent memory without uniformly sacrificing retrieval quality. At 1/11 the context cost, thousands of exchanges fit within a single prompt while the verbatim source remains available for drill-down. We release the implementation and analysis pipeline as open-source software.

Related papers

• PARSER: Read in Parallel, Reason in Depth for Long-Context LLM Agents [85.82624962221026]
  We introduce PARSER, which decouples reading from reasoning.<n>A bank of lightweight subagents each bound to a single chunk read the entire document in parallel.<n>On multi-hop QA with contexts ranging from 7K to 896K tokens, PARSER with a 4B backbone outperforms the strongest sequential memory baseline.
  arXiv  Detail & Related papers  (2026-09-06T16:19:01Z)
• Paritok-4B: Intent-Conditioned Context Compression for Coding Agents [0.9689471757238836]
  Paritok-4B is a 4B LoRA compressor for coding-agent trajectories built on two commitments.<n>It selects spans rather than rewriting them, and 96.4% of the identifiers, paths, and numbers it emits already appear in its input.<n>It is intent-conditioned: told the agent's current task, it acts chiefly inside a retained segment, selecting which lines survive.
  arXiv  Detail & Related papers  (2026-08-25T07:53:30Z)
• VisDocAgentBench: Benchmarking Agents for Visually Rich Document Retrieval [65.89247522243959]
  VisDocAgentBench is a closed-corpus benchmark comparing static and agentic retrieval under a ranked-output contract.<n>It contains 2,375 pages from 100 documents and 120 unique-target queries balanced across direct, one-bridge, and two-bridge evidence structures.<n>A strong late-interaction visual retriever reaches 97.50% Recall@1 on direct items but 2.50% on two-bridge items, exposing the limits of query--target matching when relevance depends on corpus context.
  arXiv  Detail & Related papers  (2026-08-18T15:23:06Z)
• When Your Agent Opens the Chat App: Agent-Controlled Search over Raw Chat Logs Rivals Structured Memory [21.87512744606154]
  We present ReFind, an agent-controlled search interface that builds no semantic structure at all.<n>It leaves the conversation archive unmodified, indexes it lexically at turn granularity, and combines a generic iterative keyword-search loop with four chat-native controls.<n>It attains the highest mean accuracy (58.2) of any system compared, above the strongest graph- and tree-based memory systems.
  arXiv  Detail & Related papers  (2026-08-13T07:10:22Z)
• Towards Retrieving Interaction Spaces for Agentic Search [62.666902610940525]
  Recent direct corpus (DCI) work shows that agents can interact with the raw corpus through shell tools such as 5.4 interaction and file reads.<n>We argue that the role of retrieval for agentic search is not just to select documents that fit in the LLM context window, but to construct an interaction space.
  arXiv  Detail & Related papers  (2026-06-05T03:47:40Z)
• TokenMizer: Graph-Structured Session Memory for Long-Horizon LLM Context Management [0.0]
  We present TokenMizer, an open-source proxy system that models large language model sessions as a typed knowledge graph.<n>It produces resume blocks averaging 78 tokens (range: 42-124) - 2x smaller than evaluated baselines.<n>Across all sessions, TokenMizer mean task recall 51.0%, decision recall 46.6%, and file recall 58.7%.
  arXiv  Detail & Related papers  (2026-06-04T16:12:28Z)
• SPEAR: Code-Augmented Agentic Prompt Optimization [27.161602978517706]
  SPEAR (SandFrame Prompt Engineer with Active Roll-back) is a free-form agentic with four tools that decides autonomously how and when to use them.<n>The Python tool is the largest single lever on complex judge tasks.
  arXiv  Detail & Related papers  (2026-05-25T19:01:10Z)
• Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction [127.64173950476702]
  We study direct corpus interaction (DCI), where an agent searches the raw corpus directly with general-purpose terminal tools.<n>This approach requires no offline indexing and adapts naturally to evolving local corpora.<n>Across IR benchmarks and end-to-end agentic search tasks, this simple setup substantially outperforms strong sparse, dense, and reranking baselines.
  arXiv  Detail & Related papers  (2026-05-03T19:13:11Z)
• MemMachine: A Ground-Truth-Preserving Memory System for Personalized AI Agents [2.541923091180284]
  Large Language Model (LLM) agents require persistent memory to maintain personalization, factual continuity, and long-horizon reasoning.<n>We present MemMachine, an open-source memory system that integrates short-term, long-term episodic, and profile memory.<n>MemMachine uses contextualized retrieval that expands nucleus matches with surrounding context, improving recall when relevant evidence spans multiple dialogue turns.
  arXiv  Detail & Related papers  (2026-04-06T16:57:06Z)
• Beyond RAG for Agent Memory: Retrieval by Decoupling and Aggregation [22.803751188961865]
  We argue retrieval should move beyond similarity matching and instead operate over latent components.<n>We propose xMemory, which builds a hierarchy of intact units and maintains a searchable high-level node organisation.
  arXiv  Detail & Related papers  (2026-02-02T12:04:58Z)
• Prometheus Mind: Retrofitting Memory to Frozen Language Models [0.0]
  Prometheus Mind retrofits memory to a frozen Qwen3-4B using 11 modular adapters (530MB, 7% overhead)<n>We develop Contrastive Direction Discovery (CDD), which finds semantic directions via minimal pairs without labeled data.<n>On Prometheus-132 (132 cases), the system achieves 94.4% retrieval on clean inputs.
  arXiv  Detail & Related papers  (2026-01-18T20:29:07Z)
• Cross-Document Topic-Aligned Chunking for Retrieval-Augmented Generation [0.0]
  Cross-Document Topic-Aligned chunking reconstructs knowledge at the corpus level.<n>It first identifies topics across documents, maps segments to each topic, and synthesizes them into unified chunks.
  arXiv  Detail & Related papers  (2025-11-08T11:45:45Z)
• TeaRAG: A Token-Efficient Agentic Retrieval-Augmented Generation Framework [62.66056331998838]
  TeaRAG is a token-efficient agentic RAG framework capable of compressing both retrieval content and reasoning steps.<n>Our reward function evaluates the knowledge sufficiency by a knowledge matching mechanism, while penalizing excessive reasoning steps.
  arXiv  Detail & Related papers  (2025-11-07T16:08:34Z)
• MemSearcher: Training LLMs to Reason, Search and Manage Memory via End-to-End Reinforcement Learning [73.27233666920618]
  We propose MemSearcher, an agent workflow that iteratively maintains a compact memory and combines the current turn with it.<n>At each turn, MemSearcher fuses the user's question with the memory to generate reasoning traces, perform search actions, and update memory to retain only information essential for solving the task.<n>We introduce multi-context GRPO, an end-to-end RL framework that jointly optimize reasoning, search strategies, and memory management of MemSearcher Agents.
  arXiv  Detail & Related papers  (2025-11-04T18:27:39Z)
• Compressing Many-Shots in In-Context Learning [61.231471139896506]
  We study an approach to improve the memory and computational efficiency of ICL inference by compressing the many-shot prompts.<n>We first show that existing prompt compression methods are ineffective for many-shot compression.<n>We propose MemCom, a layer-wise compression method.
  arXiv  Detail & Related papers  (2025-10-17T16:57:42Z)
• BRIEF-Pro: Universal Context Compression with Short-to-Long Synthesis for Fast and Accurate Multi-Hop Reasoning [86.4235795435618]
  BRIEF-Pro is a lightweight compressor that distills relevant evidence for a given query from retrieved documents into a concise summary.<n>It is trained to perform abstractive compression of extended contexts exceeding 10k words across a wide range of scenarios.<n> Experiments show that BRIEF-Pro generates more concise and relevant summaries, enhancing performance across small, large, and proprietary language models.
  arXiv  Detail & Related papers  (2025-10-15T17:57:45Z)
• Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning [53.45095336430027]
  We develop a unified framework that combines implicit retrieval and structured collaboration.<n>On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3% accuracy.<n>Results on SuperGPQA and TRQA confirm robustness across domains.
  arXiv  Detail & Related papers  (2025-09-25T14:05:55Z)
• R1-Compress: Long Chain-of-Thought Compression via Chunk Compression and Search [61.4807238517108]
  Chain-of-Thought (CoT) reasoning enhances large language models (LLMs) by enabling step-by-step problem-solving.<n>CoT's extension to Long-CoT introduces substantial computational overhead due to increased token length.<n>We propose R1-Compress, a two-stage chunk-level compression framework that preserves both local information and coherence.
  arXiv  Detail & Related papers  (2025-05-22T16:06:59Z)
• Answering Unseen Questions With Smaller Language Models Using Rationale Generation and Dense Retrieval [9.136948771060895]
  We evaluate two methods for further improvement in this setting. Both focus on combining rationales generated by a larger Language Model with longer contexts created from a multi-hop dense retrieval system. Our single best Reasoning model materially improves upon strong comparable prior baselines for unseen evaluation datasets.
  arXiv  Detail & Related papers  (2023-08-09T05:06:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.