Large Language Model as Token Compressor and Decompressor

• URL: http://arxiv.org/abs/2603.25340v1
• Date: Thu, 26 Mar 2026 11:30:44 GMT
• Title: Large Language Model as Token Compressor and Decompressor
• Abstract summary: We show that an off-the-shelf LLM can function as an excellent token compressor and decompressor.<n>We use a self-expressive autoencoding learning framework to translate long texts into a compact internal language of discrete, variable-length latent codes, termed Z-tokens.
• Score: 13.847845643427119
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we establish the novel insight that an off-the-shelf LLM can function as an excellent token compressor and decompressor. To demonstrate, we design a self-expressive autoencoding learning framework fine-tunes a pretrained LLM to translate long texts into a compact internal language of discrete, variable-length latent codes, termed Z-tokens, and to reconstruct the original text exactly from them. The resulting representation is content-adaptive: semantically dense segments receive more Z-tokens, while redundant or predictable regions are aggressively compressed, via lightweight LoRA-based adapter heads. Empirically, our method achieves up to 18 times token reduction on Wikipedia, CNN/DailyMail, HotpotQA, and Qulac-style long-query datasets, while preserving reconstruction fidelity and downstream performance. This simple yet effective design supports applications including prompt compression and autoregressive generation directly in the Z-token space, offering a potential pathway toward token-efficient long-context reasoning.

Related papers

• SimSD: Simple Speculative Decoding in Diffusion Language Models [61.33773959352141]
  Diffusion large language models (dLLMs) offer faster inference through parallel or blockwise decoding.<n>Their masked language modeling formulation remains incompatible with standard token-level speculative decoding.<n>We propose a speculative decoding algorithm for dLLMs, named SimSD, that equips dLLMs with temporally valid token-level contexts.<n>Our method achieves up to 7.46x higher decoding throughput while maintaining and even improving average generation quality.
  arXiv  Detail & Related papers  (2026-06-01T17:46:46Z)
• Compressing Sequences in the Latent Embedding Space: $K$-Token Merging for Large Language Models [35.95256971588258]
  Large Language Models incur significant computational and memory costs when processing long prompts.<n>We propose K-Token Merging, a latent-space compression framework that merges each contiguous block of K token embeddings into a single embedding.<n>Experiments show that K-Token Merging achieves up to 75% input length reduction with minimal performance degradation.
  arXiv  Detail & Related papers  (2026-04-16T15:32:45Z)
• TASTE-Streaming: Towards Streamable Text-Aligned Speech Tokenization and Embedding for Spoken Language Modeling [52.519745466772825]
  Text-speech joint spoken language modeling aims at natural and intelligent speech-based interactions.<n>TASTE produces speech tokens that align in lengths with their textual counterparts.<n>We propose TASTE-S, a streamable extension of TASTE suitable for real-time usage.
  arXiv  Detail & Related papers  (2026-03-12T18:13:48Z)
• On the Semantic and Syntactic Information Encoded in Proto-Tokens for One-Step Text Reconstruction [0.5097809301149341]
  Autoregressive large language models (LLMs) generate text token-by-token, requiring n forward passes to produce a sequence of length n.<n>Recent work shows that frozen LLMs can reconstruct hundreds of tokens from only two learned proto-tokens in a single forward pass.<n>We study what information these proto-tokens encode and how they behave under reconstruction and controlled constraints.
  arXiv  Detail & Related papers  (2026-02-20T15:54:10Z)
• CORE: Compact Object-centric REpresentations as a New Paradigm for Token Merging in LVLMs [29.08277140543501]
  We introduce CORE (Compact Object-centric REpresentations), a new paradigm for visual token compression.<n> CORE leverages an efficient segmentation decoder to generate object masks, which serve as a high-level semantic prior to guide the merging of visual tokens.<n>Experiments show that CORE not only establishes a new state-of-the-art on six authoritative benchmarks for fixed-rate compression, but also achieves dramatic efficiency gains in adaptive-rate settings.
  arXiv  Detail & Related papers  (2025-11-18T03:02:23Z)
• zip2zip: Inference-Time Adaptive Tokenization via Online Compression [27.16551923444618]
  zip2zip is a novel method for achieving context-adaptive tokenization in large language models.<n>We show that an existing LLM can be uptrained for zip2zip in 10 GPU-hours via parameter-efficient finetuning.<n>The resulting LLM performs test-time adaptation, learning to use hypertokens in unseen contexts and reducing input and output tokens by 15-40%.
  arXiv  Detail & Related papers  (2025-06-01T17:03:02Z)
• VQToken: Neural Discrete Token Representation Learning for Extreme Token Reduction in Video Large Language Models [35.38573641029626]
  We introduce the novel task of Extreme Short Token Reduction, which aims to represent entire videos using a minimal set of discrete tokens.<n>On the Extreme Short Token Reduction task, our VQToken compresses sequences to just 0.07 percent of their original length while incurring only a 0.66 percent drop in accuracy on the NextQA-MC benchmark.
  arXiv  Detail & Related papers  (2025-03-21T09:46:31Z)
• LightThinker: Thinking Step-by-Step Compression [74.34839026338342]
  We propose LightThinker, a method that enables large language models to dynamically compress intermediate thoughts during reasoning.<n>Inspired by human cognitive processes, LightThinker compresses thought steps into compact representations and discards the original reasoning chains.<n>Experiments show that LightThinker reduces peak memory usage and inference time, while maintaining competitive accuracy.
  arXiv  Detail & Related papers  (2025-02-21T16:57:22Z)
• Vision-centric Token Compression in Large Language Model [51.92055188780033]
  Vision Centric Token Compression (Vist) is a slow-fast compression framework that mirrors human reading.<n>On eleven in-context learning benchmarks, Vist achieves the same accuracy with 2.3 times fewer tokens, cutting FLOPs by 16% and memory by 50%.
  arXiv  Detail & Related papers  (2025-02-02T13:10:06Z)
• CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokens [49.569695524535454]
  We propose to represent speech with supervised semantic tokens, which are derived from a multilingual speech recognition model by inserting vector quantization into the encoder. Based on the tokens, we further propose a scalable zero-shot TTS synthesizer, CosyVoice, which consists of an LLM for text-to-token generation and a conditional flow matching model for token-to-speech synthesis.
  arXiv  Detail & Related papers  (2024-07-07T15:16:19Z)
• Training LLMs over Neurally Compressed Text [55.11828645767342]
  This paper explores the idea of training large language models (LLMs) over highly compressed text.<n>We propose Equal-Info Windows, a novel compression technique whereby text is segmented into blocks that each compress to the same bit length.<n>We demonstrate effective learning over neurally compressed text that improves with scale, and outperforms byte-level baselines by a wide margin on perplexity and inference speed benchmarks.
  arXiv  Detail & Related papers  (2024-04-04T17:48:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.