Haiku to Opus in Just 10 bits: LLMs Unlock Massive Compression Gains

• URL: http://arxiv.org/abs/2604.02343v1
• Date: Mon, 09 Feb 2026 18:52:02 GMT
• Title: Haiku to Opus in Just 10 bits: LLMs Unlock Massive Compression Gains
• Abstract summary: We introduce Question-Asking compression (QA), an interactive lossy protocol inspired by the game 'Twenty Questions'<n>On 8 benchmarks spanning math, science, and code, 10 binary questions recover 23% to 72% of the capability gap between a small and large model on standard benchmarks.<n>This is over 100x smaller than prior LLM-based compression.
• Score: 29.943032644387362
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study the compression of LLM-generated text across lossless and lossy regimes, characterizing a compression-compute frontier where more compression is possible at the cost of more compute. For lossless compression, domain-adapted LoRA adapters can improve LLM-based arithmetic coding by 2x over compression with the base LLM alone. For lossy compression, prompting a model for a succinct rewrite then applying arithmetic coding can achieve compression ratios of approximately 0.03, a 2x improvement over compressing the original response. We further introduce Question-Asking compression (QA), an interactive lossy protocol inspired by the game 'Twenty Questions'. A small model iteratively refines its response by asking yes/no questions to a stronger model, transferring exactly one bit per answer. On 8 benchmarks spanning math, science, and code, 10 binary questions recover 23% to 72% of the capability gap between a small and large model on standard benchmarks and 7% to 38% on harder benchmarks, achieving compression ratios of 0.0006 to 0.004. This is over 100x smaller than prior LLM-based compression (Deletang et al., 2024), suggesting that interactive protocols can transfer knowledge far more efficiently than transmitting full responses.

Related papers

• Nacrith: Neural Lossless Compression via Ensemble Context Modeling and High-Precision CDF Coding [0.0]
  We present Nacrith, a compression system that achieves the best compression results among the systems evaluated in this study on natural language text.<n>The system requires only 500 MB of GGUF weights and 1.2 GB VRAM per worker, running on consumer GPU.<n>On alice29 (Canterbury Corpus, 152 KB), Nacrith achieves 0.918 bits per byte (bpb), while compressing below the 0th-, 1st-, and 2nd-order Shannon entropy bounds.
  arXiv  Detail & Related papers  (2026-02-23T09:14:05Z)
• Arbitrary Ratio Feature Compression via Next Token Prediction [52.10426317889982]
  Arbitrary Ratio Feature Compression (ARFC) framework supports any compression ratio with a single model.<n>ARC is an auto-regressive model that performs compression via next-gressive prediction.<n>MoS module refines the compressed tokens by utilizing multiple compression results.<n>ERGC is integrated into the training process to preserve semantic and structural relationships during compression.
  arXiv  Detail & Related papers  (2026-02-12T02:38:57Z)
• Compressing Many-Shots in In-Context Learning [61.231471139896506]
  We study an approach to improve the memory and computational efficiency of ICL inference by compressing the many-shot prompts.<n>We first show that existing prompt compression methods are ineffective for many-shot compression.<n>We propose MemCom, a layer-wise compression method.
  arXiv  Detail & Related papers  (2025-10-17T16:57:42Z)
• R1-Compress: Long Chain-of-Thought Compression via Chunk Compression and Search [61.4807238517108]
  Chain-of-Thought (CoT) reasoning enhances large language models (LLMs) by enabling step-by-step problem-solving.<n>CoT's extension to Long-CoT introduces substantial computational overhead due to increased token length.<n>We propose R1-Compress, a two-stage chunk-level compression framework that preserves both local information and coherence.
  arXiv  Detail & Related papers  (2025-05-22T16:06:59Z)
• FineZip : Pushing the Limits of Large Language Models for Practical Lossless Text Compression [1.9699843876565526]
  FineZip is a novel text compression system that combines ideas of online memorization and dynamic context to reduce the compression time immensely. FineZip can compress the above corpus in approximately 4 hours compared to 9.5 days, a 54 times improvement over LLMZip and comparable performance.
  arXiv  Detail & Related papers  (2024-09-25T17:58:35Z)
• Lossless data compression by large models [18.747845226548456]
  6G communication speed requirement raises open question for revolutionary new ideas of data compression.<n>We show large language models (LLMs) understand data better than ever before.<n>We present LMCompress, which shatters all previous compression algorithms.
  arXiv  Detail & Related papers  (2024-06-24T03:58:11Z)
• LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression [43.048684907893104]
  This paper focuses on task-agnostic prompt compression for better generalizability and efficiency. We formulate prompt compression as a token classification problem to guarantee the faithfulness of the compressed prompt to the original one. Our approach leads to lower latency by explicitly learning the compression objective with smaller models such as XLM-RoBERTa-large and mBERT.
  arXiv  Detail & Related papers  (2024-03-19T17:59:56Z)
• Long Context Compression with Activation Beacon [22.054232261437186]
  Activation Beacon is a plug-in module for transformer-based LLMs. It targets effective, efficient, and flexible compression of long contexts. It achieves a 2x acceleration in inference time and an 8x reduction of memory costs for KV cache.
  arXiv  Detail & Related papers  (2024-01-07T11:57:40Z)
• Deep Lossy Plus Residual Coding for Lossless and Near-lossless Image Compression [85.93207826513192]
  We propose a unified and powerful deep lossy plus residual (DLPR) coding framework for both lossless and near-lossless image compression. We solve the joint lossy and residual compression problem in the approach of VAEs. In the near-lossless mode, we quantize the original residuals to satisfy a given $ell_infty$ error bound.
  arXiv  Detail & Related papers  (2022-09-11T12:11:56Z)
• PILC: Practical Image Lossless Compression with an End-to-end GPU Oriented Neural Framework [88.18310777246735]
  We propose an end-to-end image compression framework that achieves 200 MB/s for both compression and decompression with a single NVIDIA Tesla V100 GPU. Experiments show that our framework compresses better than PNG by a margin of 30% in multiple datasets.
  arXiv  Detail & Related papers  (2022-06-10T03:00:10Z)
• Towards Compact CNNs via Collaborative Compression [166.86915086497433]
  We propose a Collaborative Compression scheme, which joints channel pruning and tensor decomposition to compress CNN models. We achieve 52.9% FLOPs reduction by removing 48.4% parameters on ResNet-50 with only a Top-1 accuracy drop of 0.56% on ImageNet 2012.
  arXiv  Detail & Related papers  (2021-05-24T12:07:38Z)
• Single-path Bit Sharing for Automatic Loss-aware Model Compression [126.98903867768732]
  Single-path Bit Sharing (SBS) is able to significantly reduce computational cost while achieving promising performance. Our SBS compressed MobileNetV2 achieves 22.6x Bit-Operation (BOP) reduction with only 0.1% drop in the Top-1 accuracy.
  arXiv  Detail & Related papers  (2021-01-13T08:28:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.