Compress the Context, Keep the Commitments: A Formal Framework for Verifiable LLM Context Compression

• URL: http://arxiv.org/abs/2605.17304v1
• Date: Sun, 17 May 2026 07:42:37 GMT
• Title: Compress the Context, Keep the Commitments: A Formal Framework for Verifiable LLM Context Compression
• Abstract summary: We propose Context Codec, a commitment-level framework for compressing prompts and chat histories.<n> Context Codec represents dialogue state as typed, source-grounded semantic atoms with canonical identity, equivalence, conflict, confidence, risk, and evidence spans.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: LLM context is not just tokens; it is a set of commitments. Long-running conversations accumulate goals, constraints, decisions, preferences, tool results, retrieved evidence, artifacts, and safety boundaries that future responses must preserve. Existing context-management methods reduce length through truncation, retrieval, summarization, memory systems, or token-level prompt compression, but they rarely specify which semantic commitments must survive compression or how their preservation should be measured. We propose Context Codec, a commitment-level framework for compressing prompts and chat histories. Context Codec represents dialogue state as typed, source-grounded semantic atoms with canonical identity, equivalence, conflict, confidence, risk, and evidence spans. It separates five concerns - extraction, normalization, representation, rendering, and verification - and introduces metrics for Critical Atom Recall, Weighted Atom Recall, Commitment Density, and round-trip recoverability. It also defines a taxonomy of semantic compression errors, a concrete normalization procedure, conservative fallback rules for low-confidence and safety-critical atoms, and Context Compression Language (CCL), an ASCII-first compact rendering of canonical JSON atoms. In a small diagnostic study, CCL-Core occupies a useful middle ground between structured prose and JSON: more explicit and auditable than prose, usually more compact than JSON, and less risky than heavily minified notation. The result is not a claim that shorthand solves compression, but a framework for making context compression verifiable: compress the conversation, keep the commitments.

Related papers

• Every Time I Hire a Linguist, Inference Costs Go Down: On Linguistic Rules as Effective Prompt Compressors [13.49190771932777]
  We conduct evolutionary search over lexical, syntactic, semantic, and discourse seeds to find competitive rule combinations.<n>The resulting linguistic compressor requires no LM forward pass at deployment and uses only CPU-side processing for compression.<n>We evaluate it with a dual-path protocol to balance compression quality and reconstruction fidelity.
  arXiv  Detail & Related papers  (2026-07-28T06:33:27Z)
• SemanticZip: A Pilot Framework for Lossy Text Compression with LLMs as Semantic Decompressors [0.0]
  We study a more aggressive but explicitly lossy setting: compress text into compact codes that an LLM can expand into task-relevant meaning.<n>Unlike ordinary summarization, SemanticZip does not require byte-identical reconstruction.<n>This paper is a pilot framework, not a benchmark claim.
  arXiv  Detail & Related papers  (2026-05-23T12:14:04Z)
• Rethinking How to Remember: Beyond Atomic Facts in Lifelong LLM Agent Memory [88.81430082035617]
  LLM agents require a memory system that can faithfully store, efficiently retrieve, and deeply reason over accumulated dialogue history.<n>We propose TriMem, which maintains three coexisting representation granularities, including raw dialogue segments anchored by source identifiers for storage fidelity.
  arXiv  Detail & Related papers  (2026-05-19T15:05:06Z)
• On the Effectiveness of Context Compression for Repository-Level Tasks: An Empirical Investigation [22.803236151303846]
  Repository-level code intelligence tasks require large language models to process long, multi-file contexts.<n> Context compression mitigates these risks by condensing inputs.<n>We present the first systematic empirical study of context compression for repository-level code intelligence.
  arXiv  Detail & Related papers  (2026-04-15T11:00:17Z)
• Latent Context Compilation: Distilling Long Context into Compact Portable Memory [13.768393657432027]
  We propose Latent Context Compilation, a framework that shifts context processing from adaptation to compilation.<n>By utilizing a disposable LoRA module as a compiler, we distill long contexts into compact buffer tokens.<n> Experiments with Llama-3.1-8B demonstrate that Latent Context Compilation preserves fine-grained details and reasoning capabilities.
  arXiv  Detail & Related papers  (2026-01-31T08:38:07Z)
• Autoencoding-Free Context Compression for LLMs via Contextual Semantic Anchors [43.02557489472655]
  Current context compression methods rely on autoencoding tasks to train context-agnostic compression tokens to compress contextual semantics.<n>We propose Semantic-Anchor Compression (SAC), a novel method that shifts from autoencoding task based compression to an architecture that is equipped with this compression capability.<n>SAC consistently outperforms existing context compression methods across various compression ratios.
  arXiv  Detail & Related papers  (2025-10-10T01:42:14Z)
• CompLLM: Compression for Long Context Q&A [47.90063873976842]
  We introduce CompLLM, a soft compression technique designed for practical deployment.<n>Instead of processing the context holistically, CompLLM divides it into segments and compresses each one independently.<n>Our experiments show that with a 2x compression rate, at high context lengths CompLLM speeds up Time To First Token (TTFT) by up to 4x and reduces the KV cache size by 50%.
  arXiv  Detail & Related papers  (2025-09-23T16:49:43Z)
• UniGist: Towards General and Hardware-aligned Sequence-level Long Context Compression [86.33995240043936]
  UniGist is a sequence-level long-context compression framework for large language models.<n>It efficiently preserves context information by replacing raw tokens with special compression tokens (gists) in a fine-grained manner.<n>Our scheme also supports flexible inference by allowing the actual removal of compressed tokens, resulting in real-time memory savings.
  arXiv  Detail & Related papers  (2025-09-19T08:47:37Z)
• SCOPE: A Generative Approach for LLM Prompt Compression [7.813705327778312]
  We present a novel generative prompt compression method.<n>Unlike the existing token removal methods, our method centers at a chunking-and-summarization mechanism.<n>Our method achieves a significantly better compression quality, and higher stability than the state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-08-16T01:41:53Z)
• ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference [61.412894960600205]
  Large Language Models (LLMs) require significant GPU memory when processing long texts.<n>ChunkKV reimagines KV cache compression by treating semantic chunks as basic compression units.<n>Result: ChunkKV outperforms state-of-the-art methods by up to 8.7% in precision.
  arXiv  Detail & Related papers  (2025-02-01T03:49:47Z)
• UNComp: Can Matrix Entropy Uncover Sparsity? -- A Compressor Design from an Uncertainty-Aware Perspective [85.08718140718707]
  UNComp is an uncertainty-aware framework that uncovers sparsity patterns that can be used for adaptive compression.<n>By focusing on uncertainty to analyze the sparsity pattern in detail, UNComp reduces the KV cache size to 4.74% of the original, achieves a 6% prefill speedup, and improves throughput by 6.4x.
  arXiv  Detail & Related papers  (2024-10-04T02:32:36Z)
• Concise and Precise Context Compression for Tool-Using Language Models [60.606281074373136]
  We propose two strategies for compressing tool documentation into concise and precise summary sequences for tool-using language models. Results on API-Bank and APIBench show that our approach reaches a performance comparable to the upper-bound baseline under up to 16x compression ratio.
  arXiv  Detail & Related papers  (2024-07-02T08:17:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.