Efficient Video Diffusion with Sparse Information Transmission for Video Compression

• URL: http://arxiv.org/abs/2603.18501v1
• Date: Thu, 19 Mar 2026 05:15:29 GMT
• Title: Efficient Video Diffusion with Sparse Information Transmission for Video Compression
• Abstract summary: Video compression aims to maximize reconstruction quality with minimal distortions.<n>At ultra-lows, traditional end-to-end compression models tend to produce blurry images of poor quality.<n>We propose Diff-SIT to establish a new state-of-the-art in perceptual quality and temporal consistency.
• Score: 24.272279529510772
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video compression aims to maximize reconstruction quality with minimal bitrates. Beyond standard distortion metrics, perceptual quality and temporal consistency are also critical. However, at ultra-low bitrates, traditional end-to-end compression models tend to produce blurry images of poor perceptual quality. Besides, existing generative compression methods often treat video frames independently and show limitations in time coherence and efficiency. To address these challenges, we propose the Efficient Video Diffusion with Sparse Information Transmission (Diff-SIT), which comprises the Sparse Temporal Encoding Module (STEM) and the One-Step Video Diffusion with Frame Type Embedder (ODFTE). The STEM sparsely encodes the original frame sequence into an information-rich intermediate sequence, achieving significant bitrate savings. Subsequently, the ODFTE processes this intermediate sequence as a whole, which exploits the temporal correlation. During this process, our proposed Frame Type Embedder (FTE) guides the diffusion model to perform adaptive reconstruction according to different frame types to optimize the overall quality. Extensive experiments on multiple datasets demonstrate that Diff-SIT establishes a new state-of-the-art in perceptual quality and temporal consistency, particularly in the challenging ultra-low-bitrate regime. Code is released at https://github.com/MingdeZhou/Diff-SIT.

Related papers

• DiffVC-ONE: Diffusion-based Generative Video Compression with One-Step Video Diffusion Transformer [38.495966630021556]
  Generative video compression can recover rich visual details at lows, but simultaneously achieving high temporal consistency and low inference cost remains challenging.<n>We propose DiffVC-ONE, a diffusion-based generative video compression framework built on a one-step Video Transformer.<n>Experiments on multiple standard benchmarks demonstrate that DiffVC-ONE achieves perceptual state-of-theart perceptual quality and temporal consistency with low inference cost.
  arXiv  Detail & Related papers  (2026-08-20T19:20:05Z)
• High-Fidelity Causal Video Diffusion Models for Real-Time Ultra-Low-Bitrate Semantic Communication [43.045237067909106]
  We introduce a video diffusion model for high-fidelity, causal, and real-time video generation under ultra-low-bitrate semantic communication constraints.<n>Our framework achieves strong perceptual quality, semantic fidelity, and temporal consistency at ultra-lows ( 0.0003 bpp), outperforming classical, neural, and generative baselines in extensive quantitative, qualitative, and subjective evaluations.
  arXiv  Detail & Related papers  (2026-02-14T17:23:55Z)
• MTC-VAE: Multi-Level Temporal Compression with Content Awareness [54.85288415164888]
  Latent Video Diffusion Models (LVDMs) rely on Variational Autoencoders (VAEs) to compress videos into compact latent representations.<n>We present a technique to convert fixed compression rate VAEs into models that support multi-level temporal compression.
  arXiv  Detail & Related papers  (2026-02-01T17:08:02Z)
• DiffVC-RT: Towards Practical Real-Time Diffusion-based Perceptual Neural Video Compression [38.495966630021556]
  We present DiffVC-RT, the first framework designed to achieve real-time diffusion-based Neural Video Compression (NVC)<n>We show that DiffVC-RT achieves 80.1% perceptual savings in terms of LPIPS over VTM-17.0 on HEVC dataset with real-time encoding and decoding speeds of 206 / 30 fps for 720p videos on an NVIDIA H800 GPU.
  arXiv  Detail & Related papers  (2026-01-28T12:59:25Z)
• Autoregressive Video Autoencoder with Decoupled Temporal and Spatial Context [8.458436768725212]
  Video autoencoders compress videos into compact latent representations for efficient reconstruction.<n>We propose Autoregressive Video Autoencoder (ARVAE), which compresses and reconstructs each frame conditioned on its predecessor in an autoregressive manner.<n>ARVAE achieves superior reconstruction quality with extremely lightweight models and small-scale training data.
  arXiv  Detail & Related papers  (2025-12-12T05:40:01Z)
• VideoCompressa: Data-Efficient Video Understanding via Joint Temporal Compression and Spatial Reconstruction [55.66673587952058]
  Video understanding models are increasingly limited by the prohibitive storage and computational costs of large-scale datasets.<n>VideoCompressa is a novel framework for video data synthesis that reframes the problem as dynamic latent compression.
  arXiv  Detail & Related papers  (2025-11-24T07:07:58Z)
• State Space Prompting via Gathering and Spreading Spatio-Temporal Information for Video Understanding [50.866929044215965]
  We propose a State Space Prompting (SSP) method for video understanding.<n>SSP combines intra-frame inter-frame prompts to aggregate and propagate keytemporal information in the video.<n>Our SSP significantly outperforms existing SOTA methods by 2.76% on average.
  arXiv  Detail & Related papers  (2025-10-14T05:30:36Z)
• 4DGCPro: Efficient Hierarchical 4D Gaussian Compression for Progressive Volumetric Video Streaming [52.76837132019501]
  We introduce 4DGCPro, a novel hierarchical 4D compression framework.<n>4DGCPro facilitates real-time mobile decoding and high-quality rendering via progressive volumetric video streaming.<n>We present an end-to-end entropy-optimized training scheme.
  arXiv  Detail & Related papers  (2025-09-22T08:38:17Z)
• Rethinking Video Tokenization: A Conditioned Diffusion-based Approach [58.164354605550194]
  New tokenizer, Diffusion Conditioned-based Gene Tokenizer, replaces GAN-based decoder with conditional diffusion model.<n>We trained using only a basic MSE diffusion loss for reconstruction, along with KL term and LPIPS perceptual loss from scratch.<n>Even a scaled-down version of CDT (3$times inference speedup) still performs comparably with top baselines.
  arXiv  Detail & Related papers  (2025-03-05T17:59:19Z)
• Spatial Degradation-Aware and Temporal Consistent Diffusion Model for Compressed Video Super-Resolution [25.615935776826596]
  Due to storage and bandwidth limitations, videos transmitted over the Internet often exhibit low quality, characterized by low-resolution and compression artifacts.<n>Although video super-resolution (VSR) is an efficient video enhancing technique, existing VS methods focus less on compressed videos.<n>We propose a novel method that exploits the priors of pre-trained diffusion models for compressed VSR.
  arXiv  Detail & Related papers  (2025-02-11T08:57:45Z)
• High-Efficiency Neural Video Compression via Hierarchical Predictive Learning [27.41398149573729]
  Enhanced Deep Hierarchical Video Compression-DHVC 2.0- introduces superior compression performance and impressive complexity efficiency. Uses hierarchical predictive coding to transform each video frame into multiscale representations. Supports transmission-friendly progressive decoding, making it particularly advantageous for networked video applications in the presence of packet loss.
  arXiv  Detail & Related papers  (2024-10-03T15:40:58Z)
• Learned Video Compression via Heterogeneous Deformable Compensation Network [78.72508633457392]
  We propose a learned video compression framework via heterogeneous deformable compensation strategy (HDCVC) to tackle the problems of unstable compression performance. More specifically, the proposed algorithm extracts features from the two adjacent frames to estimate content-Neighborhood heterogeneous deformable (HetDeform) kernel offsets. Experimental results indicate that HDCVC achieves superior performance than the recent state-of-the-art learned video compression approaches.
  arXiv  Detail & Related papers  (2022-07-11T02:31:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.