Learned Rate Control for Frame-Level Adaptive Neural Video Compression via Dynamic Neural Network

• URL: http://arxiv.org/abs/2508.20709v1
• Date: Thu, 28 Aug 2025 12:27:23 GMT
• Title: Learned Rate Control for Frame-Level Adaptive Neural Video Compression via Dynamic Neural Network
• Authors: Chenhao Zhang, Wei Gao,
• Abstract summary: We propose a dynamic video compression framework designed for variable scenarios.<n>The proposed method achieves an average BD-Rate reduction of 14.8% and BD-PSNR gain of 0.47dB over state-of-the-art methods.
• Score: 8.645355715511702
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural Video Compression (NVC) has achieved remarkable performance in recent years. However, precise rate control remains a challenge due to the inherent limitations of learning-based codecs. To solve this issue, we propose a dynamic video compression framework designed for variable bitrate scenarios. First, to achieve variable bitrate implementation, we propose the Dynamic-Route Autoencoder with variable coding routes, each occupying partial computational complexity of the whole network and navigating to a distinct RD trade-off. Second, to approach the target bitrate, the Rate Control Agent estimates the bitrate of each route and adjusts the coding route of DRA at run time. To encompass a broad spectrum of variable bitrates while preserving overall RD performance, we employ the Joint-Routes Optimization strategy, achieving collaborative training of various routes. Extensive experiments on the HEVC and UVG datasets show that the proposed method achieves an average BD-Rate reduction of 14.8% and BD-PSNR gain of 0.47dB over state-of-the-art methods while maintaining an average bitrate error of 1.66%, achieving Rate-Distortion-Complexity Optimization (RDCO) for various bitrate and bitrate-constrained applications. Our code is available at https://git.openi.org.cn/OpenAICoding/DynamicDVC.

Related papers

• SCALED : Surrogate-gradient for Codec-Aware Learning of Downsampling in ABR Streaming [9.436544348188598]
  Over-the-Top (OTT) delivery now predominantly relies on Adaptive Bitrate (ABR) streaming.<n>Deep learning has spurred interest in jointly optimizing the ABR pipeline using learned resampling methods.<n>We introduce a novel framework that enables end-to-end training with real, non-differentiable codecs.
  arXiv  Detail & Related papers  (2026-01-30T10:38:35Z)
• Multi-Objective Pareto-Front Optimization for Efficient Adaptive VVC Streaming [3.81254285545374]
  This paper proposes a multi-objective framework to construct quality-monotonic, content-dependent Versatile Video Coding ladders.<n>Varying ladders are constructed under quality monotonicity constraints during adaptive streaming to ensure a consistent Quality of Experience (QoE)<n>Experiments are conducted on a large-scale UHD dataset (Inter-4K), with quality assessed using PSNR, VMAF, and XPSNR, and complexity measured via decoding time energy consumption.
  arXiv  Detail & Related papers  (2026-01-15T17:23:39Z)
• Multi-Scale Invertible Neural Network for Wide-Range Variable-Rate Learned Image Compression [90.59962443790593]
  In this paper, we present a variable-rate image compression model based on invertible transform to overcome limitations.<n> Specifically, we design a lightweight multi-scale invertible neural network, which maps the input image into multi-scale latent representations.<n> Experimental results demonstrate that the proposed method achieves state-of-the-art performance compared to existing variable-rate methods.
  arXiv  Detail & Related papers  (2025-03-27T09:08:39Z)
• CANeRV: Content Adaptive Neural Representation for Video Compression [89.35616046528624]
  We propose Content Adaptive Neural Representation for Video Compression (CANeRV)<n>CANeRV is an innovative INR-based video compression network that adaptively conducts structure optimisation based on the specific content of each video sequence.<n>We show that CANeRV can outperform both H.266/VVC and state-of-the-art INR-based video compression techniques across diverse video datasets.
  arXiv  Detail & Related papers  (2025-02-10T06:21:16Z)
• RL-RC-DoT: A Block-level RL agent for Task-Aware Video Compression [68.31184784672227]
  In modern applications such as autonomous driving, an overwhelming majority of videos serve as input for AI systems performing tasks.<n>It is therefore useful to optimize the encoder for a downstream task instead of for image quality.<n>Here, we address this challenge by controlling the Quantization Parameters (QPs) at the macro-block level to optimize the downstream task.
  arXiv  Detail & Related papers  (2025-01-21T15:36:08Z)
• VRVVC: Variable-Rate NeRF-Based Volumetric Video Compression [59.14355576912495]
  NeRF-based video has revolutionized visual media by delivering photorealistic Free-Viewpoint Video (FVV) experiences.<n>The substantial data volumes pose significant challenges for storage and transmission.<n>We propose VRVVC, a novel end-to-end joint variable-rate framework for video compression.
  arXiv  Detail & Related papers  (2024-12-16T01:28:04Z)
• HPC: Hierarchical Progressive Coding Framework for Volumetric Video [39.403294185116]
  Volumetric video based on Neural Radiance Field (NeRF) holds vast potential for various 3D applications. Current NeRF compression lacks the flexibility to adjust video quality and within a single model for various network and device capacities. We propose HPC, a novel hierarchical progressive video coding framework achieving variable using a single model.
  arXiv  Detail & Related papers  (2024-07-12T06:34:24Z)
• Progressive Learning with Visual Prompt Tuning for Variable-Rate Image Compression [60.689646881479064]
  We propose a progressive learning paradigm for transformer-based variable-rate image compression. Inspired by visual prompt tuning, we use LPM to extract prompts for input images and hidden features at the encoder side and decoder side, respectively. Our model outperforms all current variable image methods in terms of rate-distortion performance and approaches the state-of-the-art fixed image compression methods trained from scratch.
  arXiv  Detail & Related papers  (2023-11-23T08:29:32Z)
• Flexible Neural Image Compression via Code Editing [8.499248314440557]
  Neural image compression (NIC) has outperformed traditional image codecs in ratedistortion (R-D) performance. It usually requires a dedicated encoder-decoder pair for each point on R-D curve, which greatly hinders its practical deployment. We propose Code Editing, a highly flexible coding method for NIC based on semi-amortized inference and adaptive quantization.
  arXiv  Detail & Related papers  (2022-09-19T09:41:43Z)
• Deep Learning-Based Intra Mode Derivation for Versatile Video Coding [65.96100964146062]
  An intelligent intra mode derivation method is proposed in this paper, termed as Deep Learning based Intra Mode Derivation (DLIMD) The architecture of DLIMD is developed to adapt to different quantization parameter settings and variable coding blocks including non-square ones. The proposed method can achieve 2.28%, 1.74%, and 2.18% bit rate reduction on average for Y, U, and V components on the platform of Versatile Video Coding (VVC) test model.
  arXiv  Detail & Related papers  (2022-04-08T13:23:59Z)
• Learned Multi-Resolution Variable-Rate Image Compression with Octave-based Residual Blocks [15.308823742699039]
  We propose a new variable-rate image compression framework, which employs generalized octave convolutions (GoConv) and generalized octave transposed-convolutions (GoTConv) To enable a single model to operate with different bit rates and to learn multi-rate image features, a new objective function is introduced. Experimental results show that the proposed framework trained with variable-rate objective function outperforms the standard codecs such as H.265/HEVC-based BPG and state-of-the-art learning-based variable-rate methods.
  arXiv  Detail & Related papers  (2020-12-31T06:26:56Z)
• Neural Rate Control for Video Encoding using Imitation Learning [15.603639771786927]
  We apply imitation learning to learn a neural rate control policy. We show that our learned policy achieves 8.5% median reduction without sacrificing video quality.
  arXiv  Detail & Related papers  (2020-12-09T21:59:20Z)
• Efficient Adaptation of Neural Network Filter for Video Compression [10.769305738505071]
  We present an efficient finetuning methodology for neural-network filters. The fine-tuning is performed at encoder side to adapt the neural network to the specific content that is being encoded. The proposed method achieves much faster than conventional finetuning approaches.
  arXiv  Detail & Related papers  (2020-07-28T14:24:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.