AGORA: Adapter-Grounded Observation-Action Retention for Inference-Free Prompt Compression in LLM Agents

• URL: http://arxiv.org/abs/2605.26596v1
• Date: Tue, 26 May 2026 06:29:44 GMT
• Title: AGORA: Adapter-Grounded Observation-Action Retention for Inference-Free Prompt Compression in LLM Agents
• Abstract summary: We show that token-level extractive compressors are structurally inappropriate for LLM agents.<n>We introduce AGORA, an inference-free step-level compressor combining a structural prompt, an always-keep floor for format-critical and recency-critical content.
• Score: 3.9004035576487817
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The token-level extractive compressors widely used for general LM context are structurally inappropriate for LLM agents: across 17 (env, backbone, method) cells spanning two independent token-level method families, every cell collapses to mean reward <= 0.05 despite 1.3-13.3x realized compression. We name and characterize this failure mode as action-grammar destruction -- the tokens carrying action semantics (identifiers, brackets, action verbs) are exactly those self-information ranks lowest, so a general-purpose compressor reliably removes them and the environment rejects the residual. The diagnosis points to step-granularity compression. We introduce AGORA, an inference-free step-level compressor combining a structural prompt parser, an always-keep floor for format- and recency-critical content, and a 125M-parameter relevance scorer trained on counterfactual next-action-change labels (~2ms/step, zero per-step LLM toll). Across the compared inference-free and LLM-based methods, AGORA is the only one retaining >= 75% uncompressed performance in 8 of 9 cells (with the lone exception at 73%); a four-way component ablation isolates the structural floor as the dominant quality lever and the learned scorer as the source of 1.0-11.5x adaptive end-to-end compression from a single fixed keep ratio.

Related papers

• Steering Under Compression: Dose-Response, Capability Cost, and Failure Asymmetry in Quantized LLMs [0.0]
  Inference-time steering enables behavioral control of large language models without parameter modification.<n>Post-training quantization reduces memory and compute costs for deployment.<n>We systematically study activation steering under weight-only quantization.
  arXiv  Detail & Related papers  (2026-09-06T08:41:42Z)
• Every Time I Hire a Linguist, Inference Costs Go Down: On Linguistic Rules as Effective Prompt Compressors [13.49190771932777]
  We conduct evolutionary search over lexical, syntactic, semantic, and discourse seeds to find competitive rule combinations.<n>The resulting linguistic compressor requires no LM forward pass at deployment and uses only CPU-side processing for compression.<n>We evaluate it with a dual-path protocol to balance compression quality and reconstruction fidelity.
  arXiv  Detail & Related papers  (2026-07-28T06:33:27Z)
• SemanticZip: A Pilot Framework for Lossy Text Compression with LLMs as Semantic Decompressors [0.0]
  We study a more aggressive but explicitly lossy setting: compress text into compact codes that an LLM can expand into task-relevant meaning.<n>Unlike ordinary summarization, SemanticZip does not require byte-identical reconstruction.<n>This paper is a pilot framework, not a benchmark claim.
  arXiv  Detail & Related papers  (2026-05-23T12:14:04Z)
• SparseSAM: Structured Sparsification of Activations in Segment Anything Models [26.589924688727795]
  Segment Anything Model (SAM) achieves strong open-vocabulary segmentation, but its ViT-based image encoders dominate latency inference and memory.<n>We propose SparseSAM, a training-free sparsification framework that jointly accelerates attention and layers while preserving token identity.<n>Across four segmentation benchmarks, SparseSAM loses only 0.004 mIoU at a density and 0.021 mIoU at 0.3, a 2.10x reduction in accuracy loss versus token merging advances, while achieving 2x faster inference and 2.8x memory reduction.
  arXiv  Detail & Related papers  (2026-05-17T19:54:22Z)
• Semantic Register Compression in Multi-Agent LLM Cascades [0.0]
  We term semantic register compression as an observable failure mode in multi-agent cascades.<n>We quantify compression via inter-label separation in sentence-transformer embedding space.<n>These results demonstrate that semantic register compression is a measurable phenomenon in multi-agent LLM systems.
  arXiv  Detail & Related papers  (2026-05-12T01:11:12Z)
• The Extrapolation Cliff in On-Policy Distillation of Near-Deterministic Structured Outputs [52.709361620508595]
  ListOPD brings a student to in-domain with an 8B-SFT baseline at one-fifth the parameters.<n>On Amazon Fashion, three pre-registered tests--a fine-grid cliff interval test, and a small-clip cross-prediction--fall within their locked prediction windows--with the small-clip value matching the closed-form prediction below grid resolution.
  arXiv  Detail & Related papers  (2026-05-09T06:48:00Z)
• Lossless Prompt Compression via Dictionary-Encoding and In-Context Learning: Enabling Cost-Effective LLM Analysis of Repetitive Data [0.19780197369405136]
  In this paper, we demonstrate that Large Language Models can learn encoding keys in-context and perform analysis on encoded representations.<n>We present a compression algorithm that identifies repetitive patterns at multiple length scales.<n>This training-free approach works with API-based LLMs, directly addressing fundamental deployment constraints.
  arXiv  Detail & Related papers  (2026-03-19T21:05:53Z)
• Separating Constraint Compliance from Semantic Accuracy: A Novel Benchmark for Evaluating Instruction-Following Under Compression [0.0]
  Large language models (LLMs) exhibit degraded performance under prompt compression.<n>We introduce the CompressionDecay Test (CDCT), a benchmark that measures constraint compliance (CC) and semantic accuracy (SA)<n>We observe a universal U-curve pattern in constraint compliance (97.2% prevalence), with violations peaking at medium compression.
  arXiv  Detail & Related papers  (2025-12-02T13:25:48Z)
• Compressing Many-Shots in In-Context Learning [61.231471139896506]
  We study an approach to improve the memory and computational efficiency of ICL inference by compressing the many-shot prompts.<n>We first show that existing prompt compression methods are ineffective for many-shot compression.<n>We propose MemCom, a layer-wise compression method.
  arXiv  Detail & Related papers  (2025-10-17T16:57:42Z)
• R1-Compress: Long Chain-of-Thought Compression via Chunk Compression and Search [61.4807238517108]
  Chain-of-Thought (CoT) reasoning enhances large language models (LLMs) by enabling step-by-step problem-solving.<n>CoT's extension to Long-CoT introduces substantial computational overhead due to increased token length.<n>We propose R1-Compress, a two-stage chunk-level compression framework that preserves both local information and coherence.
  arXiv  Detail & Related papers  (2025-05-22T16:06:59Z)
• LoRC: Low-Rank Compression for LLMs KV Cache with a Progressive Compression Strategy [59.1298692559785]
  Key-Value ( KV) cache is crucial component in serving transformer-based autoregressive large language models (LLMs) Existing approaches to mitigate this issue include: (1) efficient attention variants integrated in upcycling stages; (2) KV cache compression at test time; and (3) KV cache compression at test time. We propose a low-rank approximation of KV weight matrices, allowing plug-in integration with existing transformer-based LLMs without model retraining. Our method is designed to function without model tuning in upcycling stages or task-specific profiling in test stages.
  arXiv  Detail & Related papers  (2024-10-04T03:10:53Z)
• Beyond Perplexity: Multi-dimensional Safety Evaluation of LLM Compression [33.45167213570976]
  We investigate the impact of model compression on four dimensions: (1) degeneration harm, i.e., bias and toxicity in generation; (2) representational harm, i.e., biases in discriminative tasks; (3) dialect bias; and(4) language modeling and downstream task performance. Our analysis reveals that compression can lead to unexpected consequences.
  arXiv  Detail & Related papers  (2024-07-06T05:56:22Z)
• In-Context Former: Lightning-fast Compressing Context for Large Language Model [48.831304302467004]
  In this paper, we propose a new approach to compress the long input contexts of Transformer-based large language models (LLMs) We use the cross-attention mechanism and a small number of learnable digest tokens to condense information from the contextual word embeddings. Experimental results indicate that our method requires only 1/32 of the floating-point operations of the baseline during compression and improves processing speed by 68 to 112 times.
  arXiv  Detail & Related papers  (2024-06-19T15:14:55Z)
• Compressing LLMs: The Truth is Rarely Pure and Never Simple [90.05366363633568]
  Knowledge-Intensive Compressed LLM BenchmarK aims to redefine the evaluation protocol for compressed Large Language Models. LLM-KICK unveils many favorable merits and unfortunate plights of current SoTA compression methods. LLM-KICK is designed to holistically access compressed LLMs' ability for language understanding, reasoning, generation, in-context retrieval, in-context summarization, etc.
  arXiv  Detail & Related papers  (2023-10-02T17:42:37Z)
• Do Compressed LLMs Forget Knowledge? An Experimental Study with Practical Implications [63.29358103217275]
  Large Language Models (LLMs) often leads to reduced performance, especially for knowledge-intensive tasks. We propose two conjectures on the nature of the damage: one is certain knowledge being forgotten (or erased) after compression. We introduce a variant called Inference-time Dynamic Prompting (IDP) that can effectively increase prompt diversity without incurring any inference overhead.
  arXiv  Detail & Related papers  (2023-10-02T03:12:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.