Context Compression Is Not One Thing: Readable Symbolic Re-expression vs. Coherent Summary at Matched Budget

• URL: http://arxiv.org/abs/2606.14875v1
• Date: Fri, 12 Jun 2026 18:27:23 GMT
• Title: Context Compression Is Not One Thing: Readable Symbolic Re-expression vs. Coherent Summary at Matched Budget
• Abstract summary: We study context compression for multi-hop question answering with small language models.<n>We propose Telegraph English, a readable symbolic format that rewrites retrieved passages into structured entity-relation statements.<n>In controlled experiments on MuSiQue, TwoWiki, and HotpotQA, Telegraph English outperforms three matched-budget compression baselines.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study context compression for multi-hop question answering with small language models. We propose Telegraph English, a readable symbolic format that rewrites retrieved passages into structured entity-relation statements, preserving reasoning evidence at lower token cost. In controlled experiments on MuSiQue, TwoWiki, and HotpotQA, Telegraph English outperforms three matched-budget compression baselines (character-level deletion, truncation, and random sub-sampling) on every dataset, with gains of 13 to 20 F1 percentage point. It also outperforms a coherent prose summary produced by the same encoder on the hardest dataset. A pre-registered depth-interaction hypothesis is null: the advantage does not grow with reasoning depth within datasets. We interpret these results as evidence that readable symbolic re-expression preserves entity content more densely than either natural language or coherent summarization at matched token budget.

Related papers

• Telegraph English: Semantic Prompt Compression via Structured Symbolic Rewriting [0.0]
  We introduce a prompt-compression protocol that rewrites natural language into a symbol-rich, formally-structured dialect.<n>A consequence of the line-structure rule is that compression and semantic chunking become the same operation.<n>We evaluate TE on 4,081 question-answer pairs from LongBench-v2 across five OpenAI models and two difficulty levels.
  arXiv  Detail & Related papers  (2026-05-06T02:40:55Z)
• Why Mean Pooling Works: Quantifying Second-Order Collapse in Text Embeddings [30.943998879066857]
  Mean pooling, which averages token embeddings, is the standard approach to constructing text embeddings.<n>This paper examines whether mean pooling actually works well in real models.
  arXiv  Detail & Related papers  (2026-04-30T04:09:26Z)
• From Similarity to Structure: Training-free LLM Context Compression with Hybrid Graph Priors [25.32575172623139]
  Long-context large language models remain computationally expensive to run and often fail to reliably process very long inputs.<n>We propose a training-free and model-agnostic compression framework that selects a compact set of sentences guided by structural graph priors.<n>Our method constructs a sparse hybrid sentence graph that combines mutual k-NN semantic edges with short-range sequential edges.
  arXiv  Detail & Related papers  (2026-04-25T12:40:54Z)
• ImgCoT: Compressing Long Chain of Thought into Compact Visual Tokens for Efficient Reasoning of Large Language Model [34.90582960625524]
  Long chains of thought (CoT) into compact latent tokens is crucial for efficient reasoning with large language models (LLMs)<n>We propose ImgCoT that replaces the reconstruction target from textual CoT to the visual CoT obtained by rendering CoT into images.<n>This substitutes linguistic bias with spatial inductive bias, enabling latent tokens to better capture global reasoning structure.
  arXiv  Detail & Related papers  (2026-01-30T09:06:45Z)
• Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning [53.57895922042783]
  Large Language Models (LLMs) excel at reasoning and planning when trained on chainof-thought (CoT) data.<n>We propose a hybrid representation of the reasoning process, where we partially abstract away the initial reasoning steps using latent discrete tokens.
  arXiv  Detail & Related papers  (2025-02-05T15:33:00Z)
• Context-Aware Hierarchical Merging for Long Document Summarization [56.96619074316232]
  We propose different approaches to enrich hierarchical merging with context from the source document.<n> Experimental results on datasets representing legal and narrative domains show that contextual augmentation consistently outperforms zero-shot and hierarchical merging baselines.
  arXiv  Detail & Related papers  (2025-02-03T01:14:31Z)
• Coarse-to-Fine Highlighting: Reducing Knowledge Hallucination in Large Language Models [58.952782707682815]
  COFT is a novel method to focus on different-level key texts, thereby avoiding getting lost in lengthy contexts. Experiments on the knowledge hallucination benchmark demonstrate the effectiveness of COFT, leading to a superior performance over $30%$ in the F1 score metric.
  arXiv  Detail & Related papers  (2024-10-19T13:59:48Z)
• Unpacking Tokenization: Evaluating Text Compression and its Correlation with Model Performance [34.641079276516926]
  We argue for the theoretical importance of compression, that can be viewed as 0-gram language modeling. We demonstrate the empirical importance of compression for downstream success of pre-trained language models. We show that there is a correlation between tokenizers' compression and models' downstream performance.
  arXiv  Detail & Related papers  (2024-03-10T17:02:53Z)
• A Condense-then-Select Strategy for Text Summarization [53.10242552203694]
  We propose a novel condense-then-select framework for text summarization. Our framework helps to avoid the loss of salient information, while preserving the high efficiency of sentence-level compression.
  arXiv  Detail & Related papers  (2021-06-19T10:33:10Z)
• Pareto Probing: Trading Off Accuracy for Complexity [87.09294772742737]
  We argue for a probe metric that reflects the fundamental trade-off between probe complexity and performance. Our experiments with dependency parsing reveal a wide gap in syntactic knowledge between contextual and non-contextual representations.
  arXiv  Detail & Related papers  (2020-10-05T17:27:31Z)
• Extractive Summarization as Text Matching [123.09816729675838]
  This paper creates a paradigm shift with regard to the way we build neural extractive summarization systems. We formulate the extractive summarization task as a semantic text matching problem. We have driven the state-of-the-art extractive result on CNN/DailyMail to a new level (44.41 in ROUGE-1)
  arXiv  Detail & Related papers  (2020-04-19T08:27:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.