LLM-based Source Code Compression via Thresholded Symbol Ranking

• URL: http://arxiv.org/abs/2607.24192v2
• Date: Wed, 29 Jul 2026 11:01:57 GMT
• Title: LLM-based Source Code Compression via Thresholded Symbol Ranking
• Abstract summary: General-purpose compressors (e.g., zstd, bzip2) offer a good trade-off between compression ratio and speed, but fail to exploit all special regularities inherent in source code.<n>Recent approaches leverage Large Language Models (LLMs) within Shannon's symbol-ranking framework.<n>We introduce $T$-bounded compressors deploying two novel symbol-ranking variants that bound predictions to the top-$T$ ranks.
• Score: 1.133608012542551
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: We study the problem of lossless compression of source code, motivated by the storage demands of large-scale software archives, such as Software Heritage (https://www.softwareheritage.org/). General-purpose compressors (e.g., zstd, bzip2) offer a good trade-off between compression ratio and speed, but fail to exploit all special regularities inherent in source code. Recent approaches leverage Large Language Models (LLMs) within Shannon's symbol-ranking framework, relying on a scheme in which the predicted rank can grow arbitrarily. While effective at reducing space, this setting incurs significant throughput degradation, and leaves open the question whether it is necessary to explicitly encode all ranks. In this work, we introduce LLM-based compressors deploying two novel symbol-ranking variants that bound predictions to the top-$T$ ranks ($T=1$ or $63$), with out-of-threshold symbols stored as exceptions and compressed jointly with the rank stream via general-purpose compressors. We conduct the first large-scale evaluation of LLM-based source code compression across 30 LLMs, including general-domain, code-specialized, and quantized models. Our $T$-bounded approach outperforms prior LLM-based compressors both in compression ratio (up to 37% relative improvement) and compression throughput (40% faster). Compared to general-purpose compressors (e.g., zstd, bzip2), we obtain up to 82% relative compression gain but at a lower speed, thus offering a new trade-off point in the compression-speed spectrum. We also show that these gains are stronger on source code than on natural language, suggesting an interesting indication, namely that source code exposes regularities captured by LLMs but missed by general-purpose exact-match-based compressors. We conclude by commenting on open problems that offer theoretical and practical avenues of research.

Related papers

• End-to-End Context Compression at Scale [81.70601323130997]
  Long-context language model inference is bottlenecked by memory, as the KV cache grows with context length.<n>Recent techniques to compress the KV cache fall short: they either degrade model quality substantially or require considerable time and compute to compress a single long prompt.<n>Existing approaches are not competitive with KV cache compression on the accuracy-efficiency frontier.
  arXiv  Detail & Related papers  (2026-06-08T15:43:16Z)
• OpenZL: Using Graphs to Compress Smaller and Faster [1.9508265730898475]
  We present OpenZL, a new compression strategy for data-intensive applications.<n>OpenZL implements a new theoretical framework for representing compression as a directed acyclic graph of modular codecs.<n>We show that OpenZL achieves superior compression ratios and speeds compared to state-of-the-art general-purpose compressors.
  arXiv  Detail & Related papers  (2026-05-11T03:28:15Z)
• Arbitrary Ratio Feature Compression via Next Token Prediction [52.10426317889982]
  Arbitrary Ratio Feature Compression (ARFC) framework supports any compression ratio with a single model.<n>ARC is an auto-regressive model that performs compression via next-gressive prediction.<n>MoS module refines the compressed tokens by utilizing multiple compression results.<n>ERGC is integrated into the training process to preserve semantic and structural relationships during compression.
  arXiv  Detail & Related papers  (2026-02-12T02:38:57Z)
• Cmprsr: Abstractive Token-Level Question-Agnostic Prompt Compressor [36.57824786347272]
  We present the first comprehensive LLM-as-a-compressor benchmark spanning 25 open- and closed-source models.<n>We improve the performance of the best overall vanilla compressor, with Textgrad-based compression meta-prompt optimization.<n>We call the resulting model Cmprsr and demonstrate its superiority over both extractive and vanilla abstractive compression.
  arXiv  Detail & Related papers  (2025-11-15T16:28:03Z)
• Compressing Many-Shots in In-Context Learning [61.231471139896506]
  We study an approach to improve the memory and computational efficiency of ICL inference by compressing the many-shot prompts.<n>We first show that existing prompt compression methods are ineffective for many-shot compression.<n>We propose MemCom, a layer-wise compression method.
  arXiv  Detail & Related papers  (2025-10-17T16:57:42Z)
• OpenZL: A Graph-Based Model for Compression [1.9508265730898475]
  Application-specific compressor systems outperform even the best generic compressors.<n>We show that these challenges can be overcome with a new compression strategy.<n>OpenZL compresses data into a self-describing wire format, any configuration of which can be decompressed by a universal decoder.
  arXiv  Detail & Related papers  (2025-10-03T17:40:29Z)
• CODEPROMPTZIP: Code-specific Prompt Compression for Retrieval-Augmented Generation in Coding Tasks with LMs [6.936336826531964]
  Retrieval-Augmented Generation (RAG) enhances coding tasks by incorporating retrieved code examples into prompts.<n>Existing prompt compression techniques focus on natural language, lacking tailored solutions for code.<n>We propose CodePromptZip, a framework that compresses code examples before integrating into RAG.
  arXiv  Detail & Related papers  (2025-02-19T23:15:23Z)
• Large Language Models for Lossless Image Compression: Next-Pixel Prediction in Language Space is All You Need [53.584140947828004]
  Language large model (LLM) with unprecedented intelligence is a general-purpose lossless compressor for various data modalities. We propose P$2$-LLM, a next-pixel prediction-based LLM, which integrates various elaborated insights and methodologies. Experiments on benchmark datasets demonstrate that P$2$-LLM can beat SOTA classical and learned codecs.
  arXiv  Detail & Related papers  (2024-11-19T12:15:40Z)
• Compressing LLMs: The Truth is Rarely Pure and Never Simple [90.05366363633568]
  Knowledge-Intensive Compressed LLM BenchmarK aims to redefine the evaluation protocol for compressed Large Language Models. LLM-KICK unveils many favorable merits and unfortunate plights of current SoTA compression methods. LLM-KICK is designed to holistically access compressed LLMs' ability for language understanding, reasoning, generation, in-context retrieval, in-context summarization, etc.
  arXiv  Detail & Related papers  (2023-10-02T17:42:37Z)
• Unrolled Compressed Blind-Deconvolution [77.88847247301682]
  sparse multichannel blind deconvolution (S-MBD) arises frequently in many engineering applications such as radar/sonar/ultrasound imaging. We propose a compression method that enables blind recovery from much fewer measurements with respect to the full received signal in time.
  arXiv  Detail & Related papers  (2022-09-28T15:16:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.