Prompt Compression in Production Task Orchestration: A Pre-Registered Randomized Trial

• URL: http://arxiv.org/abs/2603.23525v1
• Date: Fri, 06 Mar 2026 21:47:34 GMT
• Title: Prompt Compression in Production Task Orchestration: A Pre-Registered Randomized Trial
• Abstract summary: The economics of prompt compression depend not only on reducing input tokens but on how compression changes output length.<n>We evaluate this in a pre-registered six-arm randomized controlled trial of prompt compression on production multi-agent task-orchestration.
• Score: 0.05586191108738562
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The economics of prompt compression depend not only on reducing input tokens but on how compression changes output length, which is typically priced several times higher. We evaluate this in a pre-registered six-arm randomized controlled trial of prompt compression on production multi-agent task-orchestration, analyzing 358 successful Claude Sonnet 4.5 runs (59-61 per arm) drawn from a randomized corpus of 1,199 real orchestration instructions. We compare an uncompressed control with three uniform retention rates (r=0.8, 0.5, 0.2) and two structure-aware strategies (entropy-adaptive and recency-weighted), measuring total inference cost (input+output) and embedding-based response similarity. Moderate compression (r=0.5) reduced mean total cost by 27.9%, while aggressive compression (r=0.2) increased mean cost by 1.8% despite substantial input reduction, consistent with small mean output expansion (1.03x vs. control) and heavy-tailed uncertainty. Recency-weighted compression achieved 23.5% savings and, together with moderate compression, occupied the empirical cost-similarity Pareto frontier, whereas aggressive compression was dominated on both cost and similarity. These results show that "compress more" is not a reliable production heuristic and that output tokens must be treated as a first-class outcome when designing compression policies.

Related papers

• Token Reduction Is Not Cost Reduction [0.0]
  We show that prompt-cache traffic dominated cost composition, accounting for about 87% of reconstructed four-component cost.<n>Local payload reduction was not a reliable predictor of end-to-end billed cost.<n>We propose evaluating context-reduction systems by success-adjusted billed cost rather than token reduction alone.
  arXiv  Detail & Related papers  (2026-07-13T21:10:22Z)
• Output-Space Allocation Costs for Calibration-Guided LLM Compression: An Empirical Study [19.413175066072302]
  Training-free compression methods for large language models (LLMs) often use calibration data to guide compression decisions.<n>ROCKET, a recent method combining sparse-dictionary factorization with multi-choice knapsack problem (MCKP) allocation, derives its per-layer factorization from an output reconstruction objective.<n>We investigate whether aligning the allocation cost with the output-space objective improves compressed model fidelity.
  arXiv  Detail & Related papers  (2026-06-26T07:15:17Z)
• Prune-then-Quantize or Quantize-then-Prune? Understanding the Impact of Compression Order in Joint Model Compression [16.625053009148306]
  Joint model compression is a powerful strategy to achieve higher efficiency by combining multiple methods such as pruning and quantization.<n>A central but underexplored factor in joint model compression is the compression order, or the sequence of different methods within the compression pipeline.
  arXiv  Detail & Related papers  (2026-03-19T02:40:23Z)
• Arbitrary Ratio Feature Compression via Next Token Prediction [52.10426317889982]
  Arbitrary Ratio Feature Compression (ARFC) framework supports any compression ratio with a single model.<n>ARC is an auto-regressive model that performs compression via next-gressive prediction.<n>MoS module refines the compressed tokens by utilizing multiple compression results.<n>ERGC is integrated into the training process to preserve semantic and structural relationships during compression.
  arXiv  Detail & Related papers  (2026-02-12T02:38:57Z)
• Towards Efficient Large Language Reasoning Models via Extreme-Ratio Chain-of-Thought Compression [55.63153956934198]
  Chain-of-Thought (CoT) reasoning successfully enhances the reasoning capabilities of Large Language Models (LLMs)<n>Existing CoT compression methods often suffer from a critical loss of logical fidelity at high compression ratios.<n>We propose a novel EXTreme-RAtio Chain-of-Thought Compression framework, termed Extra-CoT, which aggressively reduces the token budget while preserving answer accuracy.
  arXiv  Detail & Related papers  (2026-02-09T06:57:15Z)
• Compressing Many-Shots in In-Context Learning [61.231471139896506]
  We study an approach to improve the memory and computational efficiency of ICL inference by compressing the many-shot prompts.<n>We first show that existing prompt compression methods are ineffective for many-shot compression.<n>We propose MemCom, a layer-wise compression method.
  arXiv  Detail & Related papers  (2025-10-17T16:57:42Z)
• Token Compensator: Altering Inference Cost of Vision Transformer without Re-Tuning [63.43972993473501]
  Token compression expedites the training and inference of Vision Transformers (ViTs) However, when applied to downstream tasks, compression degrees are mismatched between training and inference stages. We propose a model arithmetic framework to decouple the compression degrees between the two stages.
  arXiv  Detail & Related papers  (2024-08-13T10:36:43Z)
• Unbiased Compression Saves Communication in Distributed Optimization: When and How Much? [22.701976655513043]
  Communication compression can alleviate communication overhead by transmitting compressed gradients and model parameters. It is unclear whether communication compression reduces the total communication cost. We show that using independent unbiased compression can reduce the total communication cost by a factor of up to $Theta(sqrtminn, kappa)$ when all local smoothness constants are constrained.
  arXiv  Detail & Related papers  (2023-05-25T17:51:23Z)
• Towards Compact CNNs via Collaborative Compression [166.86915086497433]
  We propose a Collaborative Compression scheme, which joints channel pruning and tensor decomposition to compress CNN models. We achieve 52.9% FLOPs reduction by removing 48.4% parameters on ResNet-50 with only a Top-1 accuracy drop of 0.56% on ImageNet 2012.
  arXiv  Detail & Related papers  (2021-05-24T12:07:38Z)
• On Biased Compression for Distributed Learning [55.89300593805943]
  We show for the first time that biased compressors can lead to linear convergence rates both in the single node and distributed settings. We propose several new biased compressors with promising theoretical guarantees and practical performance.
  arXiv  Detail & Related papers  (2020-02-27T19:52:24Z)
• Uncertainty Principle for Communication Compression in Distributed and Federated Learning and the Search for an Optimal Compressor [5.09755285351264]
  We consider an unbiased compression method inspired by the Kashin representation of vectors, which we call em Kashin compression (KC). KC enjoys a em dimension independent variance bound for which we derive an explicit formula even in the regime when only a few bits need to be communicate per each vector entry.
  arXiv  Detail & Related papers  (2020-02-20T17:20:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.