CAVEWOMAN: How Large Language Models Behave Under Linguistic Input and Output Compression

• URL: http://arxiv.org/abs/2606.24083v1
• Date: Tue, 23 Jun 2026 02:56:48 GMT
• Title: CAVEWOMAN: How Large Language Models Behave Under Linguistic Input and Output Compression
• Abstract summary: We present Cavewoman, a two-channel evaluation protocol that scores every generation on task accuracy, realized per-item cost, and reference-text agreement.<n>We evaluate eight models on five datasets at five reduction levels, with both channels measured on the same items.
• Score: 77.41779716950387
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: "Talk short. Drop grammar. Save token." This caveman style is widely promoted as a way to cut inference cost, but whether it actually saves anything depends on which channel (the user's prompt or the model's response) is being compressed. We present Cavewoman, a two-channel evaluation protocol that scores every generation on task accuracy, realized per-item cost, and reference-text agreement against the model's unconstrained reference. We evaluate eight models on five datasets at five reduction levels, with both channels measured on the same items. Output compression cuts realized cost on most API models (1.4-2.4x per model, up to 3x in the best case) and on all four open-weight models under public-tier pricing. Input compression has the opposite effect, a strict lose-lose: it raises net cost rather than lowering it (~1.15x on the five-benchmark mean, up to 1.8x on the worst dataset and 2.7x under stronger compression), because models compensate with longer responses even as accuracy collapses. Under the same setting, surface text diverges from the unconstrained reference: on the non-reasoning models, roughly half of all generations are correct yet their surface text no longer entails the model's own unconstrained baseline generation. The divergence survives length-controlled re-scoring, multiple-comparisons correction, and replication under complementary semantic measures. Code and data are available at https://github.com/danielle34/cavewoman.

Related papers

• Heard but Not Heeded: Paralinguistic Information Encoding and Loss in Audio-Language Models [58.79935033105039]
  We present a mechanistic analysis of paralinguistic information in four open source models, Whisper-large-v2, Qwen2-Audio-7B Instruct, Qwen2.5- Omni-7B, and Chroma-4B.<n>We combine centered kernel alignment, linear probing with leave one speaker out evaluation, open ended tone prediction, and a content prosody leakage metric to trace how style information moves from the audio encoder to the final output.
  arXiv  Detail & Related papers  (2026-09-01T05:03:33Z)
• Kathleen Writes: Autoregressive Generation and Data Scaling Without Attention [0.0]
  We show that a byte-level, attention-free architecture can match strong baselines on classification at 450-700K parameters, without pretraining.<n>We introduce FORM DISTANCE, a non-parametric, gaming-resistant instrument for "reads like text"<n>We also report four architectural additions that did not help, and a computed lexicon reaching 94% of a learned table's top-1 accuracy at one fifth of the parameters.
  arXiv  Detail & Related papers  (2026-08-05T10:44:26Z)
• Capacity, Not Format: Rethinking Structured Reasoning Failures [0.0]
  Prior work treats structured output as a reasoning tax, but this framing is incomplete.<n>Using information-matched prose controls and a four-level schema gradient, we separate format-specific effects from prompt-length confounds.<n>We find that structured formats are capacity-dependent.
  arXiv  Detail & Related papers  (2026-06-08T12:26:54Z)
• The Format Tax [30.128781228264216]
  We find that structured output requirements substantially degrade reasoning and writing performance across open-weight models.<n>This diagnosis points to a simple principle: decouple reasoning from formats.<n>Most recent closed-weight models show little to no format tax, suggesting the problem is not inherent to structured generation but a gap that current open-weight models have yet to close.
  arXiv  Detail & Related papers  (2026-04-04T07:16:28Z)
• Moirai 2.0: When Less Is More for Time Series Forecasting [91.36760228926214]
  Moirai 2.0 is a decoder-only foundation model trained on a new corpus of 36M series.<n>It ranks among the top pretrained models while achieving a strong trade-off between accuracy, speed, and model size.<n>In terms of efficiency and model size, Moirai 2.0 is twice as fast and thirty times smaller than its prior best version, Moirai 1.0-Large.
  arXiv  Detail & Related papers  (2025-11-12T12:15:35Z)
• The Best of Both Worlds: Integrating Language Models and Diffusion Models for Video Generation [53.837937703425794]
  LanDiff is a hybrid framework that synergizes the strengths of autoregressive language models and diffusion models.<n>Our architecture introduces three key innovations: (1) a semantic tokenizer that compresses 3D visual features into compact 1D representations through efficient semantic compression; (2) a language model that generates semantic tokens with high-level semantic relationships; and (3) a streaming diffusion model that refines coarse semantics into high-fidelity videos.
  arXiv  Detail & Related papers  (2025-03-06T16:53:14Z)
• Just CHOP: Embarrassingly Simple LLM Compression [27.64461490974072]
  Large language models (LLMs) enable unparalleled few- and zero-shot reasoning capabilities but at a high computational footprint. We show that simple layer pruning coupled with an extended language model pretraining produces state-of-the-art results against structured and even semi-structured compression of models at a 7B scale. We also show how distillation, which has been super effective in task-agnostic compression of smaller BERT-style models, becomes inefficient against our simple pruning technique.
  arXiv  Detail & Related papers  (2023-05-24T08:18:35Z)
• Speculative Decoding with Big Little Decoder [108.95187338417541]
  Big Little Decoder (BiLD) is a framework that can improve inference efficiency and latency for a wide range of text generation applications. On an NVIDIA T4 GPU, our framework achieves a speedup of up to 2.12x speedup with minimal generation quality degradation. Our framework is fully plug-and-play and can be applied without any modifications in the training process or model architecture.
  arXiv  Detail & Related papers  (2023-02-15T18:55:29Z)
• CrAM: A Compression-Aware Minimizer [103.29159003723815]
  We propose a new compression-aware minimizer dubbed CrAM that modifies the optimization step in a principled way. CrAM produces dense models that can be more accurate than the standard SGD/Adam-based baselines, but which are stable under weight pruning. CrAM can produce sparse models which perform well for transfer learning, and it also works for semi-structured 2:4 pruning patterns supported by GPU hardware.
  arXiv  Detail & Related papers  (2022-07-28T16:13:28Z)
• Overfitting for Fun and Profit: Instance-Adaptive Data Compression [20.764189960709164]
  Neural data compression has been shown to outperform classical methods in terms of $RD$ performance. In this paper we take this concept to the extreme, adapting the full model to a single video, and sending model updates along with the latent representation. We demonstrate that full-model adaptation improves $RD$ performance by 1 dB, with respect to encoder-only finetuning.
  arXiv  Detail & Related papers  (2021-01-21T15:58:58Z)
• Reliable Model Compression via Label-Preservation-Aware Loss Functions [14.368823297066276]
  We present a framework that uses a teacher-student learning paradigm to better preserve labels. We obtain a significant reduction of up to 4.1X in the number of mismatches between the compressed and reference models.
  arXiv  Detail & Related papers  (2020-12-03T00:00:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.