Adaptive Rate Control for Deep Video Compression with Rate-Distortion Prediction

• URL: http://arxiv.org/abs/2412.18834v1
• Date: Wed, 25 Dec 2024 08:42:23 GMT
• Title: Adaptive Rate Control for Deep Video Compression with Rate-Distortion Prediction
• Authors: Bowen Gu, Hao Chen, Ming Lu, Jie Yao, Zhan Ma,
• Abstract summary: We propose a neural network-based $lambda$-domain rate control scheme for deep video compression.<n>The content-aware scheme is able to mitigate inter-frame quality fluctuations and adapt to abrupt changes in video content.
• Score: 28.99369130279806
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep video compression has made significant progress in recent years, achieving rate-distortion performance that surpasses that of traditional video compression methods. However, rate control schemes tailored for deep video compression have not been well studied. In this paper, we propose a neural network-based $\lambda$-domain rate control scheme for deep video compression, which determines the coding parameter $\lambda$ for each to-be-coded frame based on the rate-distortion-$\lambda$ (R-D-$\lambda$) relationships directly learned from uncompressed frames, achieving high rate control accuracy efficiently without the need for pre-encoding. Moreover, this content-aware scheme is able to mitigate inter-frame quality fluctuations and adapt to abrupt changes in video content. Specifically, we introduce two neural network-based predictors to estimate the relationship between bitrate and $\lambda$, as well as the relationship between distortion and $\lambda$ for each frame. Then we determine the coding parameter $\lambda$ for each frame to achieve the target bitrate. Experimental results demonstrate that our approach achieves high rate control accuracy at the mini-GOP level with low time overhead and mitigates inter-frame quality fluctuations across video content of varying resolutions.

Related papers

• Generative Latent Coding for Ultra-Low Bitrate Image and Video Compression [61.500904231491596]
  Most approaches for image and video compression perform transform coding in the pixel space to reduce redundancy.<n>We propose textbfGenerative textbfLatent textbfCoding (textbfGLC) models for image and video compression, GLC-image and GLC-Video.
  arXiv  Detail & Related papers  (2025-05-22T03:31:33Z)
• Embedding Compression Distortion in Video Coding for Machines [67.97469042910855]
  Currently, video transmission serves not only the Human Visual System (HVS) for viewing but also machine perception for analysis. We propose a Compression Distortion Embedding (CDRE) framework, which extracts machine-perception-related distortion representation and embeds it into downstream models. Our framework can effectively boost the rate-task performance of existing codecs with minimal overhead in terms of execution time, and number of parameters.
  arXiv  Detail & Related papers  (2025-03-27T13:01:53Z)
• Multi-Scale Invertible Neural Network for Wide-Range Variable-Rate Learned Image Compression [90.59962443790593]
  In this paper, we present a variable-rate image compression model based on invertible transform to overcome limitations. Specifically, we design a lightweight multi-scale invertible neural network, which maps the input image into multi-scale latent representations. Experimental results demonstrate that the proposed method achieves state-of-the-art performance compared to existing variable-rate methods.
  arXiv  Detail & Related papers  (2025-03-27T09:08:39Z)
• High-Efficiency Neural Video Compression via Hierarchical Predictive Learning [27.41398149573729]
  Enhanced Deep Hierarchical Video Compression-DHVC 2.0- introduces superior compression performance and impressive complexity efficiency. Uses hierarchical predictive coding to transform each video frame into multiscale representations. Supports transmission-friendly progressive decoding, making it particularly advantageous for networked video applications in the presence of packet loss.
  arXiv  Detail & Related papers  (2024-10-03T15:40:58Z)
• Accelerating Learned Video Compression via Low-Resolution Representation Learning [18.399027308582596]
  We introduce an efficiency-optimized framework for learned video compression that focuses on low-resolution representation learning. Our method achieves performance levels on par with the low-decay P configuration of the H.266 reference software VTM.
  arXiv  Detail & Related papers  (2024-07-23T12:02:57Z)
• IBVC: Interpolation-driven B-frame Video Compression [68.18440522300536]
  B-frame video compression aims to adopt bi-directional motion estimation and motion compensation (MEMC) coding for middle frame reconstruction. Previous learned approaches often directly extend neural P-frame codecs to B-frame relying on bi-directional optical-flow estimation. We propose a simple yet effective structure called Interpolation-B-frame Video Compression (IBVC) to address these issues.
  arXiv  Detail & Related papers  (2023-09-25T02:45:51Z)
• Sandwiched Video Compression: Efficiently Extending the Reach of Standard Codecs with Neural Wrappers [11.968545394054816]
  We propose a video compression system that wraps neural networks around a standard video. Networks are trained jointly to optimize a rate-distortion loss function. We observe 30% improvements in rate at the same quality over HEVC.
  arXiv  Detail & Related papers  (2023-03-20T22:03:44Z)
• Coarse-to-fine Deep Video Coding with Hyperprior-guided Mode Prediction [50.361427832256524]
  We propose a coarse-to-fine (C2F) deep video compression framework for better motion compensation. Our C2F framework can achieve better motion compensation results without significantly increasing bit costs.
  arXiv  Detail & Related papers  (2022-06-15T11:38:53Z)
• Convex Hull Prediction for Adaptive Video Streaming by Recurrent Learning [38.574550778712236]
  We propose a deep learning based method of content aware convex hull prediction. We employ a recurrent convolutional network (RCN) to implicitly analyze the complexity of video shots in order to predict their convex hulls. Our proposed model better approximations of the optimal convex hulls, and offers competitive time savings as compared to existing approaches.
  arXiv  Detail & Related papers  (2022-06-10T05:11:02Z)
• Instance-Adaptive Video Compression: Improving Neural Codecs by Training on the Test Set [14.89208053104896]
  We introduce a video compression algorithm based on instance-adaptive learning. On each video sequence to be transmitted, we finetune a pretrained compression model. We show that it enables a competitive performance even after reducing the network size by 70%.
  arXiv  Detail & Related papers  (2021-11-19T16:25:34Z)
• Ultra-low bitrate video conferencing using deep image animation [7.263312285502382]
  We propose a novel deep learning approach for ultra-low video compression for video conferencing applications. We employ deep neural networks to encode motion information as keypoint displacement and reconstruct the video signal at the decoder side.
  arXiv  Detail & Related papers  (2020-12-01T09:06:34Z)
• Conditional Entropy Coding for Efficient Video Compression [82.35389813794372]
  We propose a very simple and efficient video compression framework that only focuses on modeling the conditional entropy between frames. We first show that a simple architecture modeling the entropy between the image latent codes is as competitive as other neural video compression works and video codecs. We then propose a novel internal learning extension on top of this architecture that brings an additional 10% savings without trading off decoding speed.
  arXiv  Detail & Related papers  (2020-08-20T20:01:59Z)
• Content Adaptive and Error Propagation Aware Deep Video Compression [110.31693187153084]
  We propose a content adaptive and error propagation aware video compression system. Our method employs a joint training strategy by considering the compression performance of multiple consecutive frames instead of a single frame. Instead of using the hand-crafted coding modes in the traditional compression systems, we design an online encoder updating scheme in our system.
  arXiv  Detail & Related papers  (2020-03-25T09:04:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.