Compressing image encoders via latent distillation

• URL: http://arxiv.org/abs/2601.05639v1
• Date: Fri, 09 Jan 2026 08:50:38 GMT
• Title: Compressing image encoders via latent distillation
• Authors: Caroline Mazini Rodrigues, Nicolas Keriven, Thomas Maugey,
• Abstract summary: Deep learning models for image compression often face practical limitations in hardware-constrained applications.<n>We propose a methodology to partially compress these networks by reducing the size of their encoders.<n>Our approach uses a simplified knowledge distillation strategy to approximate the latent space of the original models with less data and shorter training, yielding lightweight encoders from heavyweight ones.
• Score: 14.979912631427334
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep learning models for image compression often face practical limitations in hardware-constrained applications. Although these models achieve high-quality reconstructions, they are typically complex, heavyweight, and require substantial training data and computational resources. We propose a methodology to partially compress these networks by reducing the size of their encoders. Our approach uses a simplified knowledge distillation strategy to approximate the latent space of the original models with less data and shorter training, yielding lightweight encoders from heavyweight ones. We evaluate the resulting lightweight encoders across two different architectures on the image compression task. Experiments show that our method preserves reconstruction quality and statistical fidelity better than training lightweight encoders with the original loss, making it practical for resource-limited environments.

Related papers

• Ultra-Efficient Decoding for End-to-End Neural Compression and Reconstruction [3.2424997710743138]
  We develop a new compression-reconstruction framework based on incorporating low-rank representation in an autoencoder with vector quantization.<n>We demonstrate that performing a series of computationally efficient low-rank operations on the learned latent representation of images can efficiently reconstruct the data with high quality.
  arXiv  Detail & Related papers  (2025-10-01T19:42:59Z)
• Efficient Learned Image Compression Through Knowledge Distillation [6.5170521662453185]
  Learned image compression sits at the intersection of machine learning and image processing.<n>Recent research suggests that neural network-based compression models consistently outperform conventional codecs.<n>This study aims to reduce the resource requirements of neural networks used for image compression by leveraging knowledge distillation.
  arXiv  Detail & Related papers  (2025-09-12T15:59:55Z)
• Multi-Scale Invertible Neural Network for Wide-Range Variable-Rate Learned Image Compression [90.59962443790593]
  In this paper, we present a variable-rate image compression model based on invertible transform to overcome limitations.<n> Specifically, we design a lightweight multi-scale invertible neural network, which maps the input image into multi-scale latent representations.<n> Experimental results demonstrate that the proposed method achieves state-of-the-art performance compared to existing variable-rate methods.
  arXiv  Detail & Related papers  (2025-03-27T09:08:39Z)
• Pathology Image Compression with Pre-trained Autoencoders [52.208181380986524]
  Whole Slide Images in digital histopathology pose significant storage, transmission, and computational efficiency challenges.<n>Standard compression methods, such as JPEG, reduce file sizes but fail to preserve fine-grained phenotypic details critical for downstream tasks.<n>In this work, we repurpose autoencoders (AEs) designed for Latent Diffusion Models as an efficient learned compression framework for pathology images.
  arXiv  Detail & Related papers  (2025-03-14T17:01:17Z)
• CALLIC: Content Adaptive Learning for Lossless Image Compression [64.47244912937204]
  CALLIC sets a new state-of-the-art (SOTA) for learned lossless image compression.<n>We propose a content-aware autoregressive self-attention mechanism by leveraging convolutional gating operations.<n>During encoding, we decompose pre-trained layers, including depth-wise convolutions, using low-rank matrices and then adapt the incremental weights on testing image by Rate-guided Progressive Fine-Tuning (RPFT)<n>RPFT fine-tunes with gradually increasing patches that are sorted in descending order by estimated entropy, optimizing learning process and reducing adaptation time.
  arXiv  Detail & Related papers  (2024-12-23T10:41:18Z)
• Streaming Lossless Volumetric Compression of Medical Images Using Gated Recurrent Convolutional Neural Network [0.0]
  This paper introduces a hardware-friendly streaming lossless volumetric compression framework. We propose a gated recurrent convolutional neural network that combines diverse convolutional structures and fusion gate mechanisms. Our method exhibits robust generalization ability and competitive compression speed.
  arXiv  Detail & Related papers  (2023-11-27T07:19:09Z)
• Extreme Image Compression using Fine-tuned VQGANs [43.43014096929809]
  We introduce vector quantization (VQ)-based generative models into the image compression domain. The codebook learned by the VQGAN model yields a strong expressive capacity. The proposed framework outperforms state-of-the-art codecs in terms of perceptual quality-oriented metrics.
  arXiv  Detail & Related papers  (2023-07-17T06:14:19Z)
• Lossless Compression with Latent Variable Models [4.289574109162585]
  We use latent variable models, which we call 'bits back with asymmetric numeral systems' (BB-ANS) The method involves interleaving encode and decode steps, and achieves an optimal rate when compressing batches of data. We describe 'Craystack', a modular software framework which we have developed for rapid prototyping of compression using deep generative models.
  arXiv  Detail & Related papers  (2021-04-21T14:03:05Z)
• Modeling Lost Information in Lossy Image Compression [72.69327382643549]
  Lossy image compression is one of the most commonly used operators for digital images. We propose a novel invertible framework called Invertible Lossy Compression (ILC) to largely mitigate the information loss problem.
  arXiv  Detail & Related papers  (2020-06-22T04:04:56Z)
• Quantization Guided JPEG Artifact Correction [69.04777875711646]
  We develop a novel architecture for artifact correction using the JPEG files quantization matrix. This allows our single model to achieve state-of-the-art performance over models trained for specific quality settings.
  arXiv  Detail & Related papers  (2020-04-17T00:10:08Z)
• Learning End-to-End Lossy Image Compression: A Benchmark [90.35363142246806]
  We first conduct a comprehensive literature survey of learned image compression methods. We describe milestones in cutting-edge learned image-compression methods, review a broad range of existing works, and provide insights into their historical development routes. By introducing a coarse-to-fine hyperprior model for entropy estimation and signal reconstruction, we achieve improved rate-distortion performance.
  arXiv  Detail & Related papers  (2020-02-10T13:13:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.