Seq2Seq2Seq: Lossless Data Compression via Discrete Latent Transformers and Reinforcement Learning

• URL: http://arxiv.org/abs/2602.12146v1
• Date: Thu, 12 Feb 2026 16:30:55 GMT
• Title: Seq2Seq2Seq: Lossless Data Compression via Discrete Latent Transformers and Reinforcement Learning
• Authors: Mahdi Khodabandeh, Ghazal Shabani, Arash Yousefi Jordehi, Seyed Abolghasem Mirroshandel,
• Abstract summary: We propose a novel compression method based on Reinforcement Learning applied to a T5 language model architecture.<n>This approach enables the compression of data into sequences of tokens rather than traditional vector representations.<n>By leveraging the latent information within language models, our system effectively compresses data without requiring explicit content understanding.
• Score: 3.2641459166493405
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Efficient lossless compression is essential for minimizing storage costs and transmission overhead while preserving data integrity. Traditional compression techniques, such as dictionary-based and statistical methods, often struggle to optimally exploit the structure and redundancy in complex data formats. Recent advancements in deep learning have opened new avenues for compression; however, many existing approaches depend on dense vector representations that obscure the underlying token structure. To address these limitations, we propose a novel lossless compression method that leverages Reinforcement Learning applied to a T5 language model architecture. This approach enables the compression of data into sequences of tokens rather than traditional vector representations. Unlike auto-encoders, which typically encode information into continuous latent spaces, our method preserves the token-based structure, aligning more closely with the original data format. This preservation allows for higher compression ratios while maintaining semantic integrity. By training the model using an off-policy Reinforcement Learning algorithm, we optimize sequence length to minimize redundancy and enhance compression efficiency. Our method introduces an efficient and adaptive data compression system built upon advanced Reinforcement Learning techniques, functioning independently of external grammatical or world knowledge. This approach shows significant improvements in compression ratios compared to conventional methods. By leveraging the latent information within language models, our system effectively compresses data without requiring explicit content understanding, paving the way for more robust and practical compression solutions across various applications.

Related papers

• Arbitrary Ratio Feature Compression via Next Token Prediction [52.10426317889982]
  Arbitrary Ratio Feature Compression (ARFC) framework supports any compression ratio with a single model.<n>ARC is an auto-regressive model that performs compression via next-gressive prediction.<n>MoS module refines the compressed tokens by utilizing multiple compression results.<n>ERGC is integrated into the training process to preserve semantic and structural relationships during compression.
  arXiv  Detail & Related papers  (2026-02-12T02:38:57Z)
• Proxy Compression for Language Modeling [58.904023114033954]
  proxy compression is an alternative training scheme that preserves the efficiency benefits of compressed inputs.<n>Experiments on code language modeling demonstrate that proxy compression substantially improves training efficiency.<n>As model scale increases, proxy-trained models eventually match or rival tokenizer approaches.
  arXiv  Detail & Related papers  (2026-02-04T07:36:46Z)
• Context Compression via Explicit Information Transmission [25.078241611630585]
  Long-context inference with Large Language Models (LLMs) is costly due to quadratic attention and growing key-value caches.<n>We propose ComprExIT, a lightweight framework that formulates soft compression into a new paradigm.
  arXiv  Detail & Related papers  (2026-02-03T17:44:12Z)
• Test-Time Steering for Lossless Text Compression via Weighted Product of Experts [27.679089540901007]
  We propose a novel framework that performs Test-Time Steering via a Weighted Product of Experts (wPoE)<n>At inference, our method adaptively combines a universal compression model with a pretrained neural language model, ensuring the compression rate is at least as good as that of the best individual model.<n>It seamlessly integrates with any autoregressive language model, providing a practical solution for enhancing text compression across diverse data distributions.
  arXiv  Detail & Related papers  (2025-11-04T16:37:56Z)
• Transformers from Compressed Representations [74.48571451824569]
  TEMPEST (TransformErs froM comPressed rEpreSenTations) is a method that exploits the inherent byte-stream structure of compressed files to design an effective tokenization and encoding strategy.<n>Our proposal substantially reduces the number of tokens required for semantic classification, thereby lowering both computational complexity and memory usage.
  arXiv  Detail & Related papers  (2025-10-26T13:48:03Z)
• UniGist: Towards General and Hardware-aligned Sequence-level Long Context Compression [86.33995240043936]
  UniGist is a sequence-level long-context compression framework for large language models.<n>It efficiently preserves context information by replacing raw tokens with special compression tokens (gists) in a fine-grained manner.<n>Our scheme also supports flexible inference by allowing the actual removal of compressed tokens, resulting in real-time memory savings.
  arXiv  Detail & Related papers  (2025-09-19T08:47:37Z)
• SCOPE: A Generative Approach for LLM Prompt Compression [7.813705327778312]
  We present a novel generative prompt compression method.<n>Unlike the existing token removal methods, our method centers at a chunking-and-summarization mechanism.<n>Our method achieves a significantly better compression quality, and higher stability than the state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-08-16T01:41:53Z)
• Compressive Meta-Learning [49.300635370079874]
  Compressive learning is a framework that enables efficient processing by using random, non-linear features.<n>We propose a framework that meta-learns both the encoding and decoding stages of compressive learning methods.<n>We explore multiple applications -- including neural network-based compressive PCA, compressive ridge regression, compressive k-means, and autoencoders.
  arXiv  Detail & Related papers  (2025-08-14T22:08:06Z)
• AlphaZip: Neural Network-Enhanced Lossless Text Compression [0.0]
  This paper introduces a lossless text compression approach using a Large Language Model (LLM) The method involves two key steps: first, prediction using a dense neural network architecture, such as a transformer block; second, compressing the predicted ranks with standard compression algorithms like Adaptive Huffman, LZ77, or Gzip.
  arXiv  Detail & Related papers  (2024-09-23T14:21:06Z)
• PowerGossip: Practical Low-Rank Communication Compression in Decentralized Deep Learning [62.440827696638664]
  We introduce a simple algorithm that directly compresses the model differences between neighboring workers. Inspired by the PowerSGD for centralized deep learning, this algorithm uses power steps to maximize the information transferred per bit.
  arXiv  Detail & Related papers  (2020-08-04T09:14:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.