The Compression Paradox in LLM Inference: Provider-Dependent Energy Effects of Prompt Compression

• URL: http://arxiv.org/abs/2603.23528v1
• Date: Fri, 06 Mar 2026 23:31:00 GMT
• Title: The Compression Paradox in LLM Inference: Provider-Dependent Energy Effects of Prompt Compression
• Abstract summary: We test whether prompt compression improves inference energy efficiency in 28,421 successful API trials.<n>For the evaluated settings, model selection and output-length control provided more consistent energy-quality tradeoffs than prompt compression.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The rapid proliferation of Large Language Models has created an environmental paradox: the very technology that could help solve climate challenges is itself becoming a significant contributor to global carbon emissions. We test whether prompt compression improves inference energy efficiency in 28,421 successful API trials (28,428 planned) across three providers (OpenAI GPT-4o-mini, Anthropic Claude-3.5-Sonnet, and DeepSeek-Chat), five benchmarks (HumanEval, MBPP, GSM8K, MATH, MMLU), and four compression ratios (r in {1.0, 0.7, 0.5, 0.3}). Energy is estimated with a token-based proxy calibrated against local direct measurements, and quality is tracked with benchmark pass rates. Compression produced substantial quality loss (overall pass rate 26.0% at baseline vs. 1.5% at r=0.7) and strongly provider-dependent energy behavior. DeepSeek exhibited output expansion under compression (21 to 798 tokens at r=0.3), corresponding to energy increases up to +2,140%, while GPT-4o-mini showed mixed effects including a reduction at r=0.5. These results indicate that input-token reduction alone is not a reliable energy optimization strategy in production inference. For the evaluated settings, model selection and output-length control provided more consistent energy-quality tradeoffs than prompt compression.

Related papers

• Compression Method Matters: Benchmark-Dependent Output Dynamics in LLM Prompt Compression [0.0]
  We present a replication and extension study of benchmark-dependent output dynamics under aggressive compression.<n>We formalize instruction survival probability (Psi), a structural metric that captures whether task-critical prompt segments remain after truncation.<n>We introduce the Compression Robustness Index (CRI) for cross-benchmark evaluation and show that single-benchmark assessments can produce misleading conclusions about compression safety and efficiency.
  arXiv  Detail & Related papers  (2026-03-06T22:39:27Z)
• Prompt Compression in Production Task Orchestration: A Pre-Registered Randomized Trial [0.05586191108738562]
  The economics of prompt compression depend not only on reducing input tokens but on how compression changes output length.<n>We evaluate this in a pre-registered six-arm randomized controlled trial of prompt compression on production multi-agent task-orchestration.
  arXiv  Detail & Related papers  (2026-03-06T21:47:34Z)
• Towards Green AI: Decoding the Energy of LLM Inference in Software Development [46.879983975894135]
  AI-assisted tools are increasingly integrated into software development, but their reliance on large language models (LLMs) introduces substantial computational and energy costs.<n>We conduct a phase-level analysis of LLM inference energy consumption, distinguishing between the (1) prefill, where the model processes the input and builds internal representations, and (2) decoding, where output tokens are generated using the stored state.
  arXiv  Detail & Related papers  (2026-02-05T14:38:19Z)
• Green MLOps: Closed-Loop, Energy-Aware Inference with NVIDIA Triton, FastAPI, and Bio-Inspired Thresholding [0.0]
  Bio-inspired framework maps protein-folding energy basins to inference cost landscapes.<n>A request is admitted only when the expected utility-to-energy trade-off is favorable.<n>Results connect biophysical energy models to Green MLORTOps and offer a practical, auditable basis for closed-loop energy-aware inference in production.
  arXiv  Detail & Related papers  (2026-01-06T15:50:11Z)
• Energy Scaling Laws for Diffusion Models: Quantifying Compute and Carbon Emissions in Image Generation [50.21021246855702]
  We propose an adaptation of Kaplan scaling laws to predict GPU energy consumption for diffusion models based on computational complexity (FLOPs)<n>Our approach decomposes diffusion model inference into text encoding, iterative denoising, and decoding components, with the hypothesis that denoising operations dominate energy consumption due to their repeated execution across multiple inference steps.<n>Our results validate the compute-bound nature of diffusion inference and provide a foundation for sustainable AI deployment planning and carbon footprint estimation.
  arXiv  Detail & Related papers  (2025-11-21T08:12:47Z)
• The Environmental Impact of Ensemble Techniques in Recommender Systems [0.0]
  Ensemble techniques in recommender systems have demonstrated accuracy improvements of 10-30%, yet their environmental impact remains unmeasured.<n>This thesis investigates how ensemble techniques influence environmental impact compared to single optimized models.
  arXiv  Detail & Related papers  (2025-11-10T14:06:58Z)
• EfficientLLM: Efficiency in Large Language Models [64.3537131208038]
  Large Language Models (LLMs) have driven significant progress, yet their growing counts and context windows incur prohibitive compute, energy, and monetary costs.<n>We introduce EfficientLLM, a novel benchmark and the first comprehensive empirical study evaluating efficiency techniques for LLMs at scale.
  arXiv  Detail & Related papers  (2025-05-20T02:27:08Z)
• Breaking the Compression Ceiling: Data-Free Pipeline for Ultra-Efficient Delta Compression [57.71917274869577]
  UltraDelta is a data-free delta compression pipeline that achieves both ultra-high compression and strong performance.<n>UltraDelta is designed to minimize redundancy, maximize information, and stabilize performance across inter-layer, intra-layer, and global dimensions.
  arXiv  Detail & Related papers  (2025-05-19T10:37:22Z)
• How Hungry is AI? Benchmarking Energy, Water, and Carbon Footprint of LLM Inference [0.0]
  This paper introduces a novel infrastructure-aware benchmarking framework for quantifying the environmental footprint of AI inference across 30 state-of-the-art models as deployed in commercial data centers.<n>Our results show that o3 and DeepSeek-R1 emerge as the most energy-intensive models, consuming over 33 Wh per long prompt, more than 70 times the consumption of GPT-4.1 nano, and that Claude-3.7 Sonnet ranks highest in eco-efficiency.<n>These findings illustrate a growing paradox: Although AI is becoming cheaper and faster, its global adoption drives disproportionate resource consumption.
  arXiv  Detail & Related papers  (2025-05-14T17:47:00Z)
• xCOMET-lite: Bridging the Gap Between Efficiency and Quality in Learned MT Evaluation Metrics [69.14652127492438]
  State-of-the-art machine translation evaluation metrics like xCOMET achieve high correlation with human judgment but rely on large encoders. We employ distillation, quantization, and pruning techniques to create efficient xCOMET alternatives. Our experiments show that, using quantization, xCOMET can be compressed up to three times with no quality degradation.
  arXiv  Detail & Related papers  (2024-06-20T17:58:34Z)
• Towards Compact CNNs via Collaborative Compression [166.86915086497433]
  We propose a Collaborative Compression scheme, which joints channel pruning and tensor decomposition to compress CNN models. We achieve 52.9% FLOPs reduction by removing 48.4% parameters on ResNet-50 with only a Top-1 accuracy drop of 0.56% on ImageNet 2012.
  arXiv  Detail & Related papers  (2021-05-24T12:07:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.