Ultra-Fast Neural Video Compression

• URL: http://arxiv.org/abs/2606.04410v1
• Date: Wed, 03 Jun 2026 03:38:05 GMT
• Title: Ultra-Fast Neural Video Compression
• Abstract summary: This paper introduces a chunk-based coding framework designed to significantly improve the rate-distortion-complexity trade-off.<n>Instead of processing frames sequentially, our approach encodes a chunk of multiple frames into a single compact latent representation and decodes them simultaneously.<n>Building on these innovations, we present DCVC-UF, a new NVC that sets a new SOTA in performance.
• Score: 29.065189275652944
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While neural video codecs (NVCs) have demonstrated superior compression ratio, their prohibitive computational complexity remains a critical barrier to real-world deployment. This paper introduces a chunk-based coding framework designed to significantly improve the rate-distortion-complexity trade-off. Instead of processing frames sequentially, our approach encodes a chunk of multiple frames into a single compact latent representation and decodes them simultaneously. This is enabled by cross-frame interaction modules for joint spatial-temporal modeling and frame-specific decoders for parallel reconstruction. This paradigm not only dramatically enhances coding throughput but also facilitates more effective modeling of long-term temporal correlations. To further boost speed, we propose a streamlined entropy coding mechanism that consolidates bit-stream interactions into a single step, substantially reducing decoding overhead. Building on these innovations, we present DCVC-UF (Ultra-Fast), a new NVC that sets a new SOTA in performance. Our experiments show that DCVC-UF can achieve ultra-fast encoding and decoding speeds, significantly outperforming previous leading codecs. DCVC-UF serves as a notable landmark in the journey of NVC evolution. The code is at https://github.com/microsoft/DCVC.

Related papers

• DiffVC-RT: Towards Practical Real-Time Diffusion-based Perceptual Neural Video Compression [38.495966630021556]
  We present DiffVC-RT, the first framework designed to achieve real-time diffusion-based Neural Video Compression (NVC)<n>We show that DiffVC-RT achieves 80.1% perceptual savings in terms of LPIPS over VTM-17.0 on HEVC dataset with real-time encoding and decoding speeds of 206 / 30 fps for 720p videos on an NVIDIA H800 GPU.
  arXiv  Detail & Related papers  (2026-01-28T12:59:25Z)
• Real-Time Neural Video Compression with Unified Intra and Inter Coding [8.998142257336674]
  We present an NVC framework with unified intra and inter coding, where every frame is processed by a single model.<n>We propose a simultaneous two-frame compression design to exploit interframe redundancy not only forwardly but also backwardly.<n>Our scheme outperforms DCVC-RT by an average of 12.1% BD-rate reduction, delivers more stable and quality per frame, and retains real-time encoding/decoding performances.
  arXiv  Detail & Related papers  (2025-10-16T08:31:44Z)
• SIEDD: Shared-Implicit Encoder with Discrete Decoders [36.705337163276255]
  Implicit Neural Representations (INRs) offer exceptional fidelity for video compression by learning per-video optimized functions.<n>Existing attempts to accelerate INR encoding often sacrifice reconstruction quality or crucial coordinate-level control.<n>We introduce SIEDD, a novel architecture that fundamentally accelerates INR encoding without these compromises.
  arXiv  Detail & Related papers  (2025-06-29T19:39:43Z)
• FastCar: Cache Attentive Replay for Fast Auto-Regressive Video Generation on the Edge [60.000984252907195]
  Auto-regressive (AR) models have recently shown promise in visual generation tasks due to their superior sampling efficiency.<n>Video generation requires a substantially larger number of tokens to produce coherent temporal frames, resulting in significant overhead during the decoding phase.<n>We propose the textbfFastCar framework to accelerate the decode phase for the AR video generation by exploring the temporal redundancy.
  arXiv  Detail & Related papers  (2025-05-17T05:00:39Z)
• STORM: Token-Efficient Long Video Understanding for Multimodal LLMs [116.4479155699528]
  STORM is a novel architecture incorporating a dedicated temporal encoder between the image encoder and the Video-LLMs.<n>We show that STORM achieves state-of-the-art results across various long video understanding benchmarks.
  arXiv  Detail & Related papers  (2025-03-06T06:17:38Z)
• Towards Practical Real-Time Neural Video Compression [60.390180067626396]
  We introduce a practical real-time neural video (NVC) designed to deliver high compression ratio, low latency and broad versatility.<n>Experiments show our proposed DCVC-RT achieves an impressive average encoding/desampling speed 125.2/112.8 (frames per second) for 1080p video, while saving an average of 21% in fps compared to H.266/VTM.
  arXiv  Detail & Related papers  (2025-02-28T06:32:23Z)
• Boosting Neural Representations for Videos with a Conditional Decoder [28.073607937396552]
  Implicit neural representations (INRs) have emerged as a promising approach for video storage and processing. This paper introduces a universal boosting framework for current implicit video representation approaches.
  arXiv  Detail & Related papers  (2024-02-28T08:32:19Z)
• VNVC: A Versatile Neural Video Coding Framework for Efficient Human-Machine Vision [59.632286735304156]
  It is more efficient to enhance/analyze the coded representations directly without decoding them into pixels. We propose a versatile neural video coding (VNVC) framework, which targets learning compact representations to support both reconstruction and direct enhancement/analysis.
  arXiv  Detail & Related papers  (2023-06-19T03:04:57Z)
• ReBotNet: Fast Real-time Video Enhancement [59.08038313427057]
  Most restoration networks are slow, have high computational bottleneck, and can't be used for real-time video enhancement. In this work, we design an efficient and fast framework to perform real-time enhancement for practical use-cases like live video calls and video streams. To evaluate our method, we emulate two new datasets that real-world video call and streaming scenarios, and show extensive results on multiple datasets where ReBotNet outperforms existing approaches with lower computations, reduced memory requirements, and faster inference time.
  arXiv  Detail & Related papers  (2023-03-23T17:58:05Z)
• Conditional Entropy Coding for Efficient Video Compression [82.35389813794372]
  We propose a very simple and efficient video compression framework that only focuses on modeling the conditional entropy between frames. We first show that a simple architecture modeling the entropy between the image latent codes is as competitive as other neural video compression works and video codecs. We then propose a novel internal learning extension on top of this architecture that brings an additional 10% savings without trading off decoding speed.
  arXiv  Detail & Related papers  (2020-08-20T20:01:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.