Group-of-Latents: Perceptual Video Compression at Extreme Bitrates via Masked Latent Generative Modeling

• URL: http://arxiv.org/abs/2607.19437v1
• Date: Tue, 21 Jul 2026 06:36:32 GMT
• Title: Group-of-Latents: Perceptual Video Compression at Extreme Bitrates via Masked Latent Generative Modeling
• Abstract summary: We propose a unified generative framework that leverages pre-trained Diffusion Transformer priors to achieve high perceptual quality at extremely lows.<n>Our method achieves state-of-the-art perceptual fidelity with rich spatial details and robust temporal consistency.
• Score: 49.11379121512358
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Most existing video compression algorithms follow a paradigm of transformation and quantization, optimizing the trade-off between distortion and bitrate. However, extremely low-bitrate compression remains an underexplored frontier where perceptual quality optimization under severely constrained coding resources has not been adequately addressed. In this paper, we propose a unified generative framework that leverages pre-trained Diffusion Transformer (DiT) priors to achieve high perceptual quality at extremely low bitrates. We first introduce a flexible Group-of-Latents (GoL) strategy within the latent space of a causal tokenizer, explicitly partitioning the latent stream into intra $I$-latents and inter $P$-latents. The Deep Compression Module (I-DCM) then encodes key $I$-latents to preserve perceptual anchors with minimal overhead. Building upon these anchors, the DiT-based Unified Latent Denoising Module (U-LDM) refines intra-frame textures and synthesizes $P$-latents from noise, reconstructing temporal dynamics at zero additional bitrate cost. Extensive experiments demonstrate that our method uniquely operates in the extreme-low-bitrate regime (e.g., (<0.005) bpp), achieving state-of-the-art perceptual fidelity with rich spatial details and robust temporal consistency. The code will be made publicly available.

Related papers

• ZeroGVC: Zero-Shot Generative Video Compression with Autoregressive Diffusion Priors [52.50325804778549]
  We propose ZeroGVC, a zero-shot generative video compression framework.<n>ZeroGVC encodes the first frame of each group of pictures (GOP) with an image and represents subsequent P-frames through Codebook-Guided Autoregressive Latent Compression.<n>Experiments on standard video compression benchmarks demonstrate that ZeroGVC achieves superior perceptual reconstruction quality at ultra-lows without any additional training.
  arXiv  Detail & Related papers  (2026-06-21T07:46:14Z)
• Dual-Constrained Diffusion Image Compression for Operational Rate-Distortion-Perception Optimization [12.089576134087594]
  We propose DCIC (Dual-Constrained Diffusion Image Compression), which integrates a learned with a diffusion-based decoder governed by joint distortion and idempotence constraints.<n>At fixed rate, dual attenuation factors $(K_D, K_P)$ jointly navigate the frontier of the distortion-perception plane, enabling continuously adjustable fidelity-realism trade-offs from a single bitstream.
  arXiv  Detail & Related papers  (2026-06-11T13:53:02Z)
• Efficient Learned Image Compression without Entropy Coding [62.01837001379442]
  We present Entropy-Coding Free Learned Image Compression (EF-LIC)<n>EF-LIC is a multi-rate framework that generates compact representation by removing statistical and correlation redundancy with low coding latency.<n>Experiments show EF-LIC achieves up to 67.86% reduction over MS-ILLM on Kodak with LPIPS.
  arXiv  Detail & Related papers  (2026-05-22T07:37:30Z)
• Geometric Transformation-Embedded Mamba for Learned Video Compression [5.89272315125701]
  We introduce a video compression framework founded on a direct transform strategy, i.e., nonlinear transform, quantization, and entropy coding.<n>Our method outperforms state-of-the-art video compression approaches in terms of perceptual quality and temporal consistency under low-bitrate constraints.
  arXiv  Detail & Related papers  (2026-03-09T03:14:07Z)
• S-PRESSO: Ultra Low Bitrate Sound Effect Compression With Diffusion Autoencoders And Offline Quantization [24.710418261668888]
  We present S-PRESSO, a 48kHz sound effect compression model that produces both continuous and discrete embeddings at ultra-lows.<n>Our model relies on a pretrained latent diffusion model to decode compressed audio embeddings learned by a latent encoder.<n>We demonstrate that S-PRESSO outperforms both continuous and discrete baselines in audio quality, acoustic similarity and reconstruction metrics.
  arXiv  Detail & Related papers  (2026-02-16T10:28:38Z)
• MTC-VAE: Multi-Level Temporal Compression with Content Awareness [54.85288415164888]
  Latent Video Diffusion Models (LVDMs) rely on Variational Autoencoders (VAEs) to compress videos into compact latent representations.<n>We present a technique to convert fixed compression rate VAEs into models that support multi-level temporal compression.
  arXiv  Detail & Related papers  (2026-02-01T17:08:02Z)
• Turbo-DDCM: Fast and Flexible Zero-Shot Diffusion-Based Image Compression [54.1069581766925]
  This paper presents an efficient zero-shot diffusion-based compression method that runs substantially faster than existing methods.<n>Our method builds upon the recently proposed Denoising Diffusion Codebook Models (DDCMs) compression scheme.<n>We introduce two flexible variants of Turbo-DDCM, a priority-aware variant that prioritizes user-specified regions and a distortion-controlled variant that compresses an image based on a target PSNR rather than a target BPP.
  arXiv  Detail & Related papers  (2025-11-09T15:41:27Z)
• StableCodec: Taming One-Step Diffusion for Extreme Image Compression [19.69733852050049]
  Diffusion-based image compression has shown remarkable potential for achieving ultra-low coding (less than 0.05 bits per pixel) with high realism.<n>Current approaches require a large number of denoising steps at the decoder to generate realistic results under extreme constraints.<n>We introduce StableCodec, which enables one-step diffusion for high-fidelity and high-realism extreme image compression.
  arXiv  Detail & Related papers  (2025-06-27T07:39:21Z)
• Single-step Diffusion for Image Compression at Ultra-Low Bitrates [19.76457078979179]
  We propose a single-step diffusion model for image compression that delivers high perceptual quality and fast decoding at ultra-lows.<n>Our approach incorporates two key innovations: (i) Vector-Quantized Residual (VQ-Residual) training, which factorizes a structural base code and a learned residual in latent space.<n>Ours achieves comparable compression performance to state-of-the-art methods while improving decoding speed by about 50x.
  arXiv  Detail & Related papers  (2025-06-19T19:53:27Z)
• Multi-Scale Invertible Neural Network for Wide-Range Variable-Rate Learned Image Compression [90.59962443790593]
  In this paper, we present a variable-rate image compression model based on invertible transform to overcome limitations.<n> Specifically, we design a lightweight multi-scale invertible neural network, which maps the input image into multi-scale latent representations.<n> Experimental results demonstrate that the proposed method achieves state-of-the-art performance compared to existing variable-rate methods.
  arXiv  Detail & Related papers  (2025-03-27T09:08:39Z)
• Once-for-All: Controllable Generative Image Compression with Dynamic Granularity Adaptation [52.82508784748278]
  This paper proposes a Control Generative Image Compression framework, termed Control-GIC.<n>Control-GIC is capable of fine-grained adaption across a broad spectrum while ensuring high-fidelity and generality compression.<n>Our experiments show that Control-GIC allows highly flexible and controllable adaption where the results demonstrate its superior performance over recent state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-06-02T14:22:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.