Flexible-Rate Learned Hierarchical Bi-Directional Video Compression With
Motion Refinement and Frame-Level Bit Allocation
- URL: http://arxiv.org/abs/2206.13613v1
- Date: Mon, 27 Jun 2022 20:18:52 GMT
- Title: Flexible-Rate Learned Hierarchical Bi-Directional Video Compression With
Motion Refinement and Frame-Level Bit Allocation
- Authors: Eren Cetin, M. Akin Yilmaz, A. Murat Tekalp
- Abstract summary: We combine motion estimation and prediction modules and compress refined residual motion vectors for improved rate-distortion performance.
We exploit the gain unit to control bit allocation among intra-coded vs. bi-directionally coded frames.
- Score: 8.80688035831646
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: This paper presents improvements and novel additions to our recent work on
end-to-end optimized hierarchical bi-directional video compression to further
advance the state-of-the-art in learned video compression. As an improvement,
we combine motion estimation and prediction modules and compress refined
residual motion vectors for improved rate-distortion performance. As novel
addition, we adapted the gain unit proposed for image compression to
flexible-rate video compression in two ways: first, the gain unit enables a
single encoder model to operate at multiple rate-distortion operating points;
second, we exploit the gain unit to control bit allocation among intra-coded
vs. bi-directionally coded frames by fine tuning corresponding models for truly
flexible-rate learned video coding. Experimental results demonstrate that we
obtain state-of-the-art rate-distortion performance exceeding those of all
prior art in learned video coding.
Related papers
- Enhancing the Rate-Distortion-Perception Flexibility of Learned Image
Codecs with Conditional Diffusion Decoders [7.485128109817576]
We show that conditional diffusion models can lead to promising results in the generative compression task when used as a decoder.
In this paper, we show that conditional diffusion models can lead to promising results in the generative compression task when used as a decoder.
arXiv Detail & Related papers (2024-03-05T11:48:35Z) - Progressive Learning with Visual Prompt Tuning for Variable-Rate Image
Compression [60.689646881479064]
We propose a progressive learning paradigm for transformer-based variable-rate image compression.
Inspired by visual prompt tuning, we use LPM to extract prompts for input images and hidden features at the encoder side and decoder side, respectively.
Our model outperforms all current variable image methods in terms of rate-distortion performance and approaches the state-of-the-art fixed image compression methods trained from scratch.
arXiv Detail & Related papers (2023-11-23T08:29:32Z) - Multi-Scale Deformable Alignment and Content-Adaptive Inference for
Flexible-Rate Bi-Directional Video Compression [8.80688035831646]
This paper proposes an adaptive motion-compensation model for end-to-end rate-distortion optimized hierarchical bi-directional video compression.
We employ a gain unit, which enables a single model to operate at multiple rate-distortion operating points.
Experimental results demonstrate state-of-the-art rate-distortion performance exceeding those of all prior art in learned video coding.
arXiv Detail & Related papers (2023-06-28T20:32:16Z) - VNVC: A Versatile Neural Video Coding Framework for Efficient
Human-Machine Vision [59.632286735304156]
It is more efficient to enhance/analyze the coded representations directly without decoding them into pixels.
We propose a versatile neural video coding (VNVC) framework, which targets learning compact representations to support both reconstruction and direct enhancement/analysis.
arXiv Detail & Related papers (2023-06-19T03:04:57Z) - Scene Matters: Model-based Deep Video Compression [13.329074811293292]
We propose a model-based video compression (MVC) framework that regards scenes as the fundamental units for video sequences.
Our proposed MVC directly models novel intensity variation of the entire video sequence in one scene, seeking non-redundant representations instead of reducing redundancy.
Our method achieves up to a 20% reduction compared to the latest video standard H.266 and is more efficient in decoding than existing video coding strategies.
arXiv Detail & Related papers (2023-03-08T13:15:19Z) - Learned Video Compression via Heterogeneous Deformable Compensation
Network [78.72508633457392]
We propose a learned video compression framework via heterogeneous deformable compensation strategy (HDCVC) to tackle the problems of unstable compression performance.
More specifically, the proposed algorithm extracts features from the two adjacent frames to estimate content-Neighborhood heterogeneous deformable (HetDeform) kernel offsets.
Experimental results indicate that HDCVC achieves superior performance than the recent state-of-the-art learned video compression approaches.
arXiv Detail & Related papers (2022-07-11T02:31:31Z) - COMISR: Compression-Informed Video Super-Resolution [76.94152284740858]
Most videos on the web or mobile devices are compressed, and the compression can be severe when the bandwidth is limited.
We propose a new compression-informed video super-resolution model to restore high-resolution content without introducing artifacts caused by compression.
arXiv Detail & Related papers (2021-05-04T01:24:44Z) - Conditional Entropy Coding for Efficient Video Compression [82.35389813794372]
We propose a very simple and efficient video compression framework that only focuses on modeling the conditional entropy between frames.
We first show that a simple architecture modeling the entropy between the image latent codes is as competitive as other neural video compression works and video codecs.
We then propose a novel internal learning extension on top of this architecture that brings an additional 10% savings without trading off decoding speed.
arXiv Detail & Related papers (2020-08-20T20:01:59Z) - End-to-End Rate-Distortion Optimization for Bi-Directional Learned Video
Compression [10.404162481860634]
Learned video compression allows end-to-end rate-distortion optimized training of all nonlinear modules.
In this paper, we propose for the first time end-to-end optimization of a hierarchical, bi-directional motion compensated by accumulating cost function over fixed-size groups of pictures.
arXiv Detail & Related papers (2020-08-11T22:50:06Z) - Content Adaptive and Error Propagation Aware Deep Video Compression [110.31693187153084]
We propose a content adaptive and error propagation aware video compression system.
Our method employs a joint training strategy by considering the compression performance of multiple consecutive frames instead of a single frame.
Instead of using the hand-crafted coding modes in the traditional compression systems, we design an online encoder updating scheme in our system.
arXiv Detail & Related papers (2020-03-25T09:04:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.