TokenMizer: Graph-Structured Session Memory for Long-Horizon LLM Context Management

• URL: http://arxiv.org/abs/2606.06337v1
• Date: Thu, 04 Jun 2026 16:12:28 GMT
• Title: TokenMizer: Graph-Structured Session Memory for Long-Horizon LLM Context Management
• Abstract summary: We present TokenMizer, an open-source proxy system that models large language model sessions as a typed knowledge graph.<n>It produces resume blocks averaging 78 tokens (range: 42-124) - 2x smaller than evaluated baselines.<n>Across all sessions, TokenMizer mean task recall 51.0%, decision recall 46.6%, and file recall 58.7%.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language model (LLM) deployments for long-horizon tasks face a fundamental constraint: context windows are finite while productive work sessions are not. When history exceeds the Maximum Effective Context Window (MECW), critical structured information - architectural decisions, task transitions, file histories - is silently discarded. Existing mitigations treat history as flat text, destroying the relational structure that makes sessions resumable. We present TokenMizer, an open-source proxy system that models LLM session history as a typed knowledge graph. The schema defines 14 node types and 7 edge types. A hybrid extraction pipeline populates the graph incrementally, while a three-tier checkpoint system serializes it into compact resume blocks. An 8-layer compression pipeline reduces context overhead, and a semantic cache reduces repeated-query latency. Evaluated on a controlled benchmark of 21 sessions spanning 5 domains, TokenMizer demonstrates significant token economy. It produces resume blocks averaging 78 tokens (range: 42-124) - 2x smaller than evaluated baselines (159-170 tokens) - while achieving higher decision recall (+9-17 percentage points). Crucially, baselines only preserve that a technology was mentioned; TokenMizer preserves the rationale. Across all sessions, TokenMizer achieves mean task recall 51.0%, decision recall 46.6%, and file recall 58.7%. Variance reflects domain heterogeneity: explicit imperative phrasing (software engineering) scores higher than implicit reasoning (research). Ablation studies show fuzzy label matching is the dominant improvement factor (+33 pp task recall). The heuristic compression achieves 47.3% token reduction with zero external dependencies. TokenMizer provides a queryable alternative to text-retention baselines at half the token cost.

Related papers

• Beyond Vector Similarity: Hierarchical Context-Aware Graph RAG vs Standard RAG in Enterprise Code Migration [3.164177271211724]
  This paper introduces a Hierarchical Context-Resident Graph (HCRG) methodology to resolve limitations.<n>Our pipeline uses tree-sitter for Abstract Syntax Tree (AST) extraction, maps architectural edges into a Google Cloud Spanner Property Graph, and serializes this structure into a Gemini Context Cache.<n>Ultimately, while trading code complexity for reduced hallucinations, Graph RAG provides a substantially more viable, architecturally sound path for automated enterprise modernization.
  arXiv  Detail & Related papers  (2026-09-11T05:54:41Z)
• Codebook Agent: Amortized Topology Design for LLM Multi-Agent Systems [67.04448659688579]
  We develop a query-independent 16-entry codebook that ranks the top decoded candidates in a single batched forward pass.<n>With no iterative search and no message passing at test time, Codebook Agent is the most accurate method on all six benchmarks.
  arXiv  Detail & Related papers  (2026-09-02T08:10:22Z)
• Context as an Environment: Programmatic Context Management for Long-Horizon Agents [47.21335299084925]
  We present Scroll, a context manager that treats each agent session as an executable Session Environment.<n>The kernel maintains a typed across model calls, allowing tool outputs, retrieved history, and derived state to be bound to variables.
  arXiv  Detail & Related papers  (2026-08-21T23:39:19Z)
• Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models [65.49950267695267]
  We introduce Omni2LoRA, a framework for efficient parametric memory compression via coherence-preserving context distillation.<n>By converting multimodal memory into a fixed-budget, reusable parameter state, our method drives answer-time multimodal-token load to zero.
  arXiv  Detail & Related papers  (2026-08-10T07:49:10Z)
• RepoReasoner: Evaluating Repository-Level Code Reasoning Ability of Long-Context Language Models [54.53236295237077]
  We introduce RepoReasoner, a benchmark for evaluating repository-level code reasoning.<n> Output Prediction measures fine-grained, stateful execution reasoning across files.<n>Call Chain Prediction evaluates high-level architectural dependency understanding under noisy context.
  arXiv  Detail & Related papers  (2026-07-28T17:12:41Z)
• Shared Selective Persistent Memory for Agentic LLM Systems [0.0]
  LLM systems generate code through multi-turn tool use.<n>Each session starts from zero, discarding the configuration choices, domain constraints, data schemas, and tool-use patterns that made previous sessions productive.<n>We introduce shared selective persistent memory, an architecture that identifies and retains four categories of reusable context.
  arXiv  Detail & Related papers  (2026-07-10T15:07:00Z)
• ContextRAG: Extraction-Free Hierarchical Graph Construction for Retrieval-Augmented Generation [0.9936176521793759]
  Graph-structured retrieval-augmented generation (RAG) systems can improve answer quality on multi-hop questions.<n>Current systems rely on large language models (LLMs) to extract entities, relations, and summaries during indexing.<n>We present ContextRAG, a graph RAG system whose graph topology is constructed without LLM-based entity or relation extraction.
  arXiv  Detail & Related papers  (2026-05-19T12:08:19Z)
• Cooperative Memory Paging with Keyword Bookmarks for Long-Horizon LLM Conversations [2.6382975801439836]
  We propose cooperative paging: segments are replaced with minimal keyword bookmarks, and the model is given a recall() tool to retrieve full content on demand.<n>On the LoCoMo benchmark, cooperative paging achieves the highest answer quality among six methods -- outperforming truncation, BM25, word-overlap retrieval, a search-tool baseline, and full context.<n>We then study the paging design space with a 5x4 ablation over boundary strategies and eviction policies (3,176 synthetic probes, 1,600 LoCoMo probes)
  arXiv  Detail & Related papers  (2026-04-14T07:06:35Z)
• Escaping the Context Bottleneck: Active Context Curation for LLM Agents via Reinforcement Learning [6.577332846728808]
  We introduce a symbiotic framework that decouples context management from task execution.<n>Our architecture pairs a lightweight, specialized policy model, ContextCurator, with a powerful frozen foundation model, TaskExecutor.<n>On WebArena, our framework improves the success rate of Gemini-3.0-flash from 36.4% to 41.2% while reducing token consumption by 8.8%.<n>On DeepSearch, it achieves a 57.1% success rate, compared with 53.9%, while reducing token consumption by a factor of 8.
  arXiv  Detail & Related papers  (2026-04-13T13:39:17Z)
• APEX-EM: Non-Parametric Online Learning for Autonomous Agents via Structured Procedural-Episodic Experience Replay [7.370176470430802]
  LLM-based autonomous agents lack persistent procedural memory.<n>We present APEX-EM, a non-parametric online learning framework that accumulates, retrieves, and reuses structured procedural plans.
  arXiv  Detail & Related papers  (2026-03-31T00:24:56Z)
• Contextual Memory Virtualisation: DAG-Based State Management and Structurally Lossless Trimming for LLM Agents [0.0]
  We propose Contextual Memory Virtualisation (CMV), a system that treats accumulated LLM understanding as version-controlled state.<n> CMV models session history as a Directed Acyclic Graph (DAG) with formally defined snapshot, branch, and trim primitives.<n>A single-user case-study evaluation shows that trimming remains economically viable under prompt caching.
  arXiv  Detail & Related papers  (2026-02-25T20:52:52Z)
• dVoting: Fast Voting for dLLMs [71.572316901001]
  Diffusion Large Language Models (dLLMs) represent a new paradigm beyond autoregressive modeling.<n>dLLMs can generate tokens at arbitrary positions in parallel, endowing them with significant potential for parallel test-time scaling.<n>We introduce dVoting, a fast voting technique that boosts reasoning capability without training, with only an acceptable extra computational overhead.
  arXiv  Detail & Related papers  (2026-02-12T16:35:05Z)
• TeaRAG: A Token-Efficient Agentic Retrieval-Augmented Generation Framework [62.66056331998838]
  TeaRAG is a token-efficient agentic RAG framework capable of compressing both retrieval content and reasoning steps.<n>Our reward function evaluates the knowledge sufficiency by a knowledge matching mechanism, while penalizing excessive reasoning steps.
  arXiv  Detail & Related papers  (2025-11-07T16:08:34Z)
• Training-Free Exponential Context Extension via Cascading KV Cache [49.608367376911694]
  We introduce a novel mechanism that leverages cascading sub-cache buffers to selectively retain the most relevant tokens.<n>Our method reduces prefill stage latency by a factor of 6.8 when compared to flash attention on 1M tokens.
  arXiv  Detail & Related papers  (2024-06-24T03:59:17Z)
• Hierarchical Context Merging: Better Long Context Understanding for Pre-trained LLMs [61.40047491337793]
  We present Hierarchical cOntext MERging (HOMER), a new training-free scheme designed to overcome the limitations of large language models. HomeR uses a divide-and-conquer algorithm, dividing long inputs into manageable chunks. A token reduction technique precedes each merging, ensuring memory usage efficiency.
  arXiv  Detail & Related papers  (2024-04-16T06:34:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.