PoC: Performance-oriented Context Compression for Large Language Models via Performance Prediction

• URL: http://arxiv.org/abs/2603.19733v1
• Date: Fri, 20 Mar 2026 08:18:46 GMT
• Title: PoC: Performance-oriented Context Compression for Large Language Models via Performance Prediction
• Abstract summary: We introduce a paradigm shift to Performance-oriented Context Compression (PoC)<n>PoC employs a lightweight performance predictor to automatically find the most aggressive compression ratio.<n>Our work paves the way for a more reliable, efficient, and performance-aware deployment of context compression for Large Language Models.
• Score: 40.705016911274
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While context compression can mitigate the growing inference costs of Large Language Models (LLMs) by shortening contexts, existing methods that specify a target compression ratio or length suffer from unpredictable performance degradation, hindering their reliable deployment. We introduce a paradigm shift to Performance-oriented Context Compression (PoC), where developers specify an acceptable performance floor instead of a compression ratio. PoC employs a lightweight performance predictor to automatically find the most aggressive compression ratio that satisfies this constraint before steering an off-the-shelf compressor. We design and compare two predictor variants: a simple context-agnostic predictor and a more sophisticated context-aware one that considers the input's inherent compressibility. On both question-answering and summarization benchmarks, the context-aware predictor consistently achieves lower performance prediction error than the context-agnostic predictor, while the resulting context-aware PoC attains a superior overall performance. Our work paves the way for a more reliable, efficient, and performance-aware deployment of context compression for LLMs.

Related papers

• Planning-aligned Token Compression for Long-Context Autonomous Driving [95.59023657139208]
  We propose a planning-aligned working memory framework built on conditional VQ-VA.<n>Compression is conditioned on both historical trajectory and a learned planning intent.<n>We evaluate on high-signal dynamic scenarios where historical context is most critical for behavior correctness.
  arXiv  Detail & Related papers  (2026-06-05T17:16:21Z)
• Thinking as Compression: Your Reasoning Model is Secretly a Context Compressor [18.235525350999335]
  This work reveals that a thinking model itself can naturally compress long contexts by organizing task-relevant information.<n>We thus derive Thinking as Compression (TaC), a new compression paradigm that treats thinking itself as compressed context.
  arXiv  Detail & Related papers  (2026-05-27T16:36:01Z)
• Density-aware Soft Context Compression with Semi-Dynamic Compression Ratio [19.375601589555327]
  We introduce Semi-Dynamic Context Compression framework.<n>Our approach features a Discrete Ratio Selector, which predicts a compression target based on intrinsic information density.<n>Our framework consistently outperforms static baselines.
  arXiv  Detail & Related papers  (2026-03-26T21:36:55Z)
• Arbitrary Ratio Feature Compression via Next Token Prediction [52.10426317889982]
  Arbitrary Ratio Feature Compression (ARFC) framework supports any compression ratio with a single model.<n>ARC is an auto-regressive model that performs compression via next-gressive prediction.<n>MoS module refines the compressed tokens by utilizing multiple compression results.<n>ERGC is integrated into the training process to preserve semantic and structural relationships during compression.
  arXiv  Detail & Related papers  (2026-02-12T02:38:57Z)
• Towards Efficient Large Language Reasoning Models via Extreme-Ratio Chain-of-Thought Compression [55.63153956934198]
  Chain-of-Thought (CoT) reasoning successfully enhances the reasoning capabilities of Large Language Models (LLMs)<n>Existing CoT compression methods often suffer from a critical loss of logical fidelity at high compression ratios.<n>We propose a novel EXTreme-RAtio Chain-of-Thought Compression framework, termed Extra-CoT, which aggressively reduces the token budget while preserving answer accuracy.
  arXiv  Detail & Related papers  (2026-02-09T06:57:15Z)
• AttnComp: Attention-Guided Adaptive Context Compression for Retrieval-Augmented Generation [27.480791258325066]
  We introduce AttnComp, an adaptive, efficient and context-aware compression framework.<n>AttnComp employs a Top-P compression algorithm to retain the minimal set of documents.<n>In addition to compression, AttnComp estimates response confidence by assessing the overall relevance of the retrieved content.
  arXiv  Detail & Related papers  (2025-09-22T08:18:50Z)
• Lag-Relative Sparse Attention In Long Context Training [8.365610885641276]
  We propose Lag-Relative Sparse Attention(LRSA) anchored by the LagKV compression method for long context post-training.<n>Our method performs chunk-by-chunk prefilling, which selects the top K most relevant key-value pairs in a fixed-size lagging window.
  arXiv  Detail & Related papers  (2025-06-13T06:49:53Z)
• Long Context In-Context Compression by Getting to the Gist of Gisting [50.24627831994713]
  GistPool is an in-context compression method with no architectural modification to the decoder transformer.<n>We demonstrate that gisting struggles with longer contexts, with significant performance drops even at minimal compression rates.<n>GistPool preserves the simplicity of gisting, while significantly boosting its performance on long context compression tasks.
  arXiv  Detail & Related papers  (2025-04-11T19:23:31Z)
• Choose Your Model Size: Any Compression of Large Language Models Without Re-Computation [10.376875638696504]
  This work presents Any Compression via Iterative Pruning (ACIP), a novel algorithmic approach to determine a compression-performance trade-off.<n>We use an SVD-reparametrization of linear layers and iteratively prune their singular values with a sparsity-inducing penalty.<n>We show that ACIP seamlessly complements common quantization-based compression techniques.
  arXiv  Detail & Related papers  (2025-02-03T18:40:58Z)
• Ranking LLMs by compression [13.801767671391604]
  We use five large language models as priors for compression, then compare their performance on challenging natural language processing tasks. Experimental results show that compression ratio and model performance are positively correlated, so it can be used as a general metric to evaluate large language models.
  arXiv  Detail & Related papers  (2024-06-20T10:23:38Z)
• Learning Accurate Performance Predictors for Ultrafast Automated Model Compression [86.22294249097203]
  We propose an ultrafast automated model compression framework called SeerNet for flexible network deployment. Our method achieves competitive accuracy-complexity trade-offs with significant reduction of the search cost.
  arXiv  Detail & Related papers  (2023-04-13T10:52:49Z)
• What do Compressed Large Language Models Forget? Robustness Challenges in Model Compression [68.82486784654817]
  We study two popular model compression techniques including knowledge distillation and pruning. We show that compressed models are significantly less robust than their PLM counterparts on adversarial test sets. We develop a regularization strategy for model compression based on sample uncertainty.
  arXiv  Detail & Related papers  (2021-10-16T00:20:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.