VoRTeC: Taming Foundation Flow for One-step Real time Video Compression

• URL: http://arxiv.org/abs/2609.02291v2
• Date: Thu, 03 Sep 2026 02:54:14 GMT
• Title: VoRTeC: Taming Foundation Flow for One-step Real time Video Compression
• Abstract summary: $mathttVoRTeC$ is a Video Compression framework built upon a foundational flow model (Wan2.1)<n>Our method reduces bit consumption by 58% compared to prior diffusion-based approaches.<n>$mathttVoRTeC$ achieves a decoding speed of 13 FPS at 720p and 32 FPS at 480p.
• Score: 38.79321243531853
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Ultra-low bitrate video compression still faces critical challenges: traditional neural video compression inevitably introduces blurring artifacts, while diffusion-based generative video compression suffers from excessive decoding latency and poor temporal consistency. To address these issues, we propose $\mathtt{VoRTeC}$, a Video Compression framework built upon a foundational flow model (Wan2.1). By compactly encoding latent video representations, predicting the positions of compressed representations along flow trajectories, and integrating multi-scale priors, $\mathtt{VoRTeC}$ enables the compressor to harness generative video flow priors effectively. Without accessing the parameters or gradients of flow matching networks, our framework achieves one-step decoding and reconstructions with high perceptual fidelity. Meanwhile, we maintain consistency across frame groups via tail-frame reuse and prior caching. Extensive experiments demonstrate that our method reduces bit consumption by 58\% compared to prior diffusion-based approaches, with decoding speed boosted by 3 to 197 times: $\mathtt{VoRTeC}$ achieves a decoding speed of 13 FPS at 720p and 32 FPS at 480p.

Related papers

• ZeroGVC: Zero-Shot Generative Video Compression with Autoregressive Diffusion Priors [52.50325804778549]
  We propose ZeroGVC, a zero-shot generative video compression framework.<n>ZeroGVC encodes the first frame of each group of pictures (GOP) with an image and represents subsequent P-frames through Codebook-Guided Autoregressive Latent Compression.<n>Experiments on standard video compression benchmarks demonstrate that ZeroGVC achieves superior perceptual reconstruction quality at ultra-lows without any additional training.
  arXiv  Detail & Related papers  (2026-06-21T07:46:14Z)
• Next-Frame Decoding for Ultra-Low-Bitrate Image Compression with Video Diffusion Priors [30.653078447499894]
  We present a novel paradigm for ultra-low-bitrate image compression (ULB-IC)<n>We define an explicit intermediate state during decoding: a compact anchor frame.<n>We then reinterpret generative decoding as a virtual temporal transition from this anchor to the final reconstructed image.
  arXiv  Detail & Related papers  (2026-03-16T11:24:26Z)
• DiffVC-RT: Towards Practical Real-Time Diffusion-based Perceptual Neural Video Compression [38.495966630021556]
  We present DiffVC-RT, the first framework designed to achieve real-time diffusion-based Neural Video Compression (NVC)<n>We show that DiffVC-RT achieves 80.1% perceptual savings in terms of LPIPS over VTM-17.0 on HEVC dataset with real-time encoding and decoding speeds of 206 / 30 fps for 720p videos on an NVIDIA H800 GPU.
  arXiv  Detail & Related papers  (2026-01-28T12:59:25Z)
• HiStream: Efficient High-Resolution Video Generation via Redundancy-Eliminated Streaming [58.55148690302855]
  HiStream is an efficient autoregressive framework that systematically reduces redundancy across three axes.<n>On 1080p benchmarks, our primary HiStream model (i+ii) achieves state-of-the-art visual quality while demonstrating up to 76.2x faster denoising compared to the Wan2.1 baseline.<n>Our faster variant, HiStream+, applies all three optimizations, achieving a 107.5x acceleration over the baseline.
  arXiv  Detail & Related papers  (2025-12-24T18:59:58Z)
• Low-Bitrate Video Compression through Semantic-Conditioned Diffusion [19.21409064179896]
  We propose a severe failure that transmits only the most meaningful information while relying on generative detail for priors for priors.<n>A conditional video reconstructs high-quality, temporally coherent videos from semantic, appearance, and motion cues.
  arXiv  Detail & Related papers  (2025-11-29T09:38:16Z)
• VideoCompressa: Data-Efficient Video Understanding via Joint Temporal Compression and Spatial Reconstruction [55.66673587952058]
  Video understanding models are increasingly limited by the prohibitive storage and computational costs of large-scale datasets.<n>VideoCompressa is a novel framework for video data synthesis that reframes the problem as dynamic latent compression.
  arXiv  Detail & Related papers  (2025-11-24T07:07:58Z)
• Accelerating Learned Video Compression via Low-Resolution Representation Learning [18.399027308582596]
  We introduce an efficiency-optimized framework for learned video compression that focuses on low-resolution representation learning. Our method achieves performance levels on par with the low-decay P configuration of the H.266 reference software VTM.
  arXiv  Detail & Related papers  (2024-07-23T12:02:57Z)
• Fast-Vid2Vid: Spatial-Temporal Compression for Video-to-Video Synthesis [40.249030338644225]
  Video-to-Video synthesis (Vid2Vid) has achieved remarkable results in generating a photo-realistic video from a sequence of semantic maps. Fast-Vid2Vid achieves around real-time performance as 20 FPS and saves around 8x computational cost on a single V100 GPU.
  arXiv  Detail & Related papers  (2022-07-11T17:57:57Z)
• Conditional Entropy Coding for Efficient Video Compression [82.35389813794372]
  We propose a very simple and efficient video compression framework that only focuses on modeling the conditional entropy between frames. We first show that a simple architecture modeling the entropy between the image latent codes is as competitive as other neural video compression works and video codecs. We then propose a novel internal learning extension on top of this architecture that brings an additional 10% savings without trading off decoding speed.
  arXiv  Detail & Related papers  (2020-08-20T20:01:59Z)
• Content Adaptive and Error Propagation Aware Deep Video Compression [110.31693187153084]
  We propose a content adaptive and error propagation aware video compression system. Our method employs a joint training strategy by considering the compression performance of multiple consecutive frames instead of a single frame. Instead of using the hand-crafted coding modes in the traditional compression systems, we design an online encoder updating scheme in our system.
  arXiv  Detail & Related papers  (2020-03-25T09:04:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.