Relevant but Incomplete: Referential Dangling as a Paradigm-Level Failure Mode in Hard Prompt Compression

• URL: http://arxiv.org/abs/2608.04569v1
• Date: Wed, 05 Aug 2026 07:58:53 GMT
• Title: Relevant but Incomplete: Referential Dangling as a Paradigm-Level Failure Mode in Hard Prompt Compression
• Abstract summary: We show that when retained text contains an answer but deleted text defines the entity needed to interpret it, we call the result referential dangling.<n>We train a compact to rank omitted sentences by whether they are needed to interpret retained text and reinsert the top-ranked candidates without support annotations at inference.
• Score: 18.435649248662944
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Hard prompt compression reduces long-context inference cost by independently scoring tokens, sentences, or chunks and retaining the highest-scoring units under a budget. We identify a structural failure in this procedure: independent selection can split dependent evidence pairs, retaining one member while deleting the other. When retained text contains an answer but deleted text defines the entity needed to interpret it, we call the result referential dangling. At a compression ratio of 0.30, Beaver, which ranks coherent chunks using Qwen3-0.6B embeddings, leaves the answer path incomplete in 34-54% of bridge examples across three multi-hop question answering datasets. On a shared HotpotQA bridge set, all six hard compressors we test exhibit dangling at rates up to 60%, and every document in LongBench-v2 Single-Document QA contains at least one dangling reference. On dangling examples evaluated with Qwen3-8B, reinserting the missing supporting paragraph while removing nonsupporting paragraphs to maintain the token budget improves accuracy by 29-34 percentage points (p < 0.0001), recovering at least 88% of the gap to contexts retaining both supporting paragraphs. Stronger answer models do not absorb the loss: on MuSiQue, GPT-5.5 is 8.8 points less accurate on compressed contexts than on contexts retaining both supporting paragraphs. Finally, we train a compact classifier to rank omitted sentences by whether they are needed to interpret retained text and reinsert the top-ranked candidates without support annotations at inference. On HotpotQA with Qwen3-8B, this automatic restoration improves accuracy by 4.7 points while changing the compression ratio only from 0.30 to 0.31. Hard compressors should optimize both relevance and referential completeness.

Related papers

• PARSER: Read in Parallel, Reason in Depth for Long-Context LLM Agents [85.82624962221026]
  We introduce PARSER, which decouples reading from reasoning.<n>A bank of lightweight subagents each bound to a single chunk read the entire document in parallel.<n>On multi-hop QA with contexts ranging from 7K to 896K tokens, PARSER with a 4B backbone outperforms the strongest sequential memory baseline.
  arXiv  Detail & Related papers  (2026-09-06T16:19:01Z)
• On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability [57.03393882933515]
  We describe the architecture and ablations of Qwen3.8-Flash-Next, a sparse mixture-of-experts model with 125B parameters, 6B activated per token, and 51B parameters of n-gram embedding tables held off the accelerator.<n>On fourteen pre-training benchmarks the model leads the 397B-A17B predecessor on eight and trails it on the rest by at most 2.6 points, at 1/3 the activated parameters, 1/3 the training tokens, and roughly 1/9 the training FLOPs.
  arXiv  Detail & Related papers  (2026-08-31T06:35:07Z)
• Scalable Question-Centric Text-to-Image Evaluation: Reliable Ranking, Fine-Grained Diagnosis, and Cost-Aware Routing [9.025294637245661]
  QC-T2I-Bench is a question-centric framework that converts open prompts into attributed atomic questions.<n>We evaluate multiple open-source T2I models on English and Chinese prompts.
  arXiv  Detail & Related papers  (2026-08-25T06:20:18Z)
• IndicQE-APE: A Benchmark for Quality Estimation and Automatic Post-Editing for Indic Languages [77.88427845181154]
  We consolidate the WMT 2020--2024 shared-task lineage with an extended English--Malayalam resource into indicqe: $126,754$ instances over nine directional pairs.<n>We benchmark six prompted LLMs and three COMET metrics on segment-level QE, and three systems on APE.
  arXiv  Detail & Related papers  (2026-08-17T09:50:52Z)
• The Sleeping Agent: What Gist-Based Context Compression Loses and Why [0.0]
  We use a biologically-inspired compression framework motivated by sleep-based memory consolidation to study when compression helps and when it hurts.<n>We find gist compression substantially outperforms truncation on multi-hop reasoning and single-hop factual questions.<n>We trace this failure to a specific mechanism: the gist abstraction prompt preserves relational and event structure while discarding dates and times.
  arXiv  Detail & Related papers  (2026-08-12T08:19:15Z)
• ShortOPD: Recovering Pruned LLMs with Short-to-Long On-Policy Distillation [72.54604107217669]
  textscpass@$k$ nearly vanishes after compression, yet textscpass@$k$ recovers substantially under repeated sampling.<n>Recovery should train on the compressed model's own on-policy states with dense token-level supervision.<n>We propose textbfshortopd, a short-to-long OPD schedule that detects teacher-confirmed repetitive suffixes.
  arXiv  Detail & Related papers  (2026-07-14T17:50:50Z)
• How Query Visibility Changes KV-Cache Compression Rankings: A Matched-Budget Audit [5.366718741784969]
  KV-cache compression methods are predominantly evaluated with the query appended to the context before compression.<n>We present a matched-budget audit of six published compression methods against three trivial baselines.
  arXiv  Detail & Related papers  (2026-07-11T09:30:35Z)
• Length Penalties Make Chain-of-Thought Less Monitorable [0.0]
  Training with length penalties does not stop misleading hints from steering models.<n>We train Qwen3-4B and Qwen3-14B variants with different target chain lengths.<n>We evaluate them with biasing-hint interventions on held-out MMLU-Pro-R and four transfer benchmarks.
  arXiv  Detail & Related papers  (2026-07-08T14:18:26Z)
• What Survives Into Context: A Diagnostic for Budget-Constrained Multi-Hop RAG and When Submodular Evidence Packing Improves It [0.0]
  We argue that document recall is the wrong quantity to optimize in a fixed reader-context budget.<n>We introduce answer-in-context, a diagnostic that measures whether a gold answer survives as a contiguous span in the packed reader context.<n>We map the scope of this win honestly: it requires the conjunction of (i) multi-hop complementary structure, (ii) retrieval that surfaces the evidence, (iii) a binding but not extreme budget, and (iv) a reader weak enough that evidence density, not reading capacity, is the bottleneck.
  arXiv  Detail & Related papers  (2026-07-01T10:12:15Z)
• Context-Aware Distillation and Ablation for Text2DSL [78.29352305480285]
  We replace prompt-only synthetic generation with context-aware distillation.<n>A teacher large language model operates under an explicitly defined structured context.<n>This scales the verified PolkitBench corpus from 4,204 to 10,073 pairs.
  arXiv  Detail & Related papers  (2026-06-21T16:27:24Z)
• Fixed RAG Compression Collapses Measured Reader Scaling [2.9089118242427627]
  We show fixed compression can raise average accuracy while hiding reader upgrades and reversing model rankings.<n>We release ragscale, a toolkit built on 177,000 row-level compression transitions to audit reader scaling with three readers in one day.
  arXiv  Detail & Related papers  (2026-06-20T00:04:31Z)
• Context Compression Is Not One Thing: Readable Symbolic Re-expression vs. Coherent Summary at Matched Budget [0.0]
  We study context compression for multi-hop question answering with small language models.<n>We propose Telegraph English, a readable symbolic format that rewrites retrieved passages into structured entity-relation statements.<n>In controlled experiments on MuSiQue, TwoWiki, and HotpotQA, Telegraph English outperforms three matched-budget compression baselines.
  arXiv  Detail & Related papers  (2026-06-12T18:27:23Z)
• ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference [61.412894960600205]
  Large Language Models (LLMs) require significant GPU memory when processing long texts.<n>ChunkKV reimagines KV cache compression by treating semantic chunks as basic compression units.<n>Result: ChunkKV outperforms state-of-the-art methods by up to 8.7% in precision.
  arXiv  Detail & Related papers  (2025-02-01T03:49:47Z)
• Span Pointer Networks for Non-Autoregressive Task-Oriented Semantic Parsing [55.97957664897004]
  An effective recipe for building seq2seq, non-autoregressive, task-orienteds to map utterances to semantic frames proceeds in three steps. These models are typically bottlenecked by length prediction. In our work, we propose non-autoregressives which shift the decoding task from text generation to span prediction.
  arXiv  Detail & Related papers  (2021-04-15T07:02:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.