Hierarchical B-frame Video Coding Using Two-Layer CANF without Motion
Coding
- URL: http://arxiv.org/abs/2304.02690v1
- Date: Wed, 5 Apr 2023 18:36:28 GMT
- Title: Hierarchical B-frame Video Coding Using Two-Layer CANF without Motion
Coding
- Authors: David Alexandre, Hsueh-Ming Hang, Wen-Hsiao Peng
- Abstract summary: We propose a novel B-frame coding architecture based on two-layer Augmented Normalization Flows (CANF)
Our proposed idea of video compression without motion coding offers a new direction for learned video coding.
The rate-distortion performance of our scheme is slightly lower than that of the state-of-the-art learned B-frame coding scheme, B-CANF, but outperforms other learned B-frame coding schemes.
- Score: 17.998825368770635
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Typical video compression systems consist of two main modules: motion coding
and residual coding. This general architecture is adopted by classical coding
schemes (such as international standards H.265 and H.266) and deep
learning-based coding schemes. We propose a novel B-frame coding architecture
based on two-layer Conditional Augmented Normalization Flows (CANF). It has the
striking feature of not transmitting any motion information. Our proposed idea
of video compression without motion coding offers a new direction for learned
video coding. Our base layer is a low-resolution image compressor that replaces
the full-resolution motion compressor. The low-resolution coded image is merged
with the warped high-resolution images to generate a high-quality image as a
conditioning signal for the enhancement-layer image coding in full resolution.
One advantage of this architecture is significantly reduced computational
complexity due to eliminating the motion information compressor. In addition,
we adopt a skip-mode coding technique to reduce the transmitted latent samples.
The rate-distortion performance of our scheme is slightly lower than that of
the state-of-the-art learned B-frame coding scheme, B-CANF, but outperforms
other learned B-frame coding schemes. However, compared to B-CANF, our scheme
saves 45% of multiply-accumulate operations (MACs) for encoding and 27% of MACs
for decoding. The code is available at https://nycu-clab.github.io.
Related papers
- Accelerating Learned Video Compression via Low-Resolution Representation Learning [18.399027308582596]
We introduce an efficiency-optimized framework for learned video compression that focuses on low-resolution representation learning.
Our method achieves performance levels on par with the low-decay P configuration of the H.266 reference software VTM.
arXiv Detail & Related papers (2024-07-23T12:02:57Z) - MISC: Ultra-low Bitrate Image Semantic Compression Driven by Large Multimodal Model [78.4051835615796]
This paper proposes a method called Multimodal Image Semantic Compression.
It consists of an LMM encoder for extracting the semantic information of the image, a map encoder to locate the region corresponding to the semantic, an image encoder generates an extremely compressed bitstream, and a decoder reconstructs the image based on the above information.
It can achieve optimal consistency and perception results while saving perceptual 50%, which has strong potential applications in the next generation of storage and communication.
arXiv Detail & Related papers (2024-02-26T17:11:11Z) - IBVC: Interpolation-driven B-frame Video Compression [68.18440522300536]
B-frame video compression aims to adopt bi-directional motion estimation and motion compensation (MEMC) coding for middle frame reconstruction.
Previous learned approaches often directly extend neural P-frame codecs to B-frame relying on bi-directional optical-flow estimation.
We propose a simple yet effective structure called Interpolation-B-frame Video Compression (IBVC) to address these issues.
arXiv Detail & Related papers (2023-09-25T02:45:51Z) - Low-complexity Deep Video Compression with A Distributed Coding
Architecture [4.5885672744218]
Prevalent predictive coding-based video compression methods rely on a heavy encoder to reduce temporal redundancy.
Traditional distributed coding methods suffer from a substantial performance gap to predictive coding ones.
We propose the first end-to-end distributed deep video compression framework to improve rate-distortion performance.
arXiv Detail & Related papers (2023-03-21T05:34:04Z) - Block Modulating Video Compression: An Ultra Low Complexity Image
Compression Encoder for Resource Limited Platforms [15.646001803032421]
An ultra low-cost image Modulating Video Compression (BMVC) is proposed to be implemented on mobile platforms with low consumption of power and computation resources.
Two types of BMVC decoders, implemented by deep neural networks, are presented.
arXiv Detail & Related papers (2022-05-07T16:20:09Z) - Conditional Entropy Coding for Efficient Video Compression [82.35389813794372]
We propose a very simple and efficient video compression framework that only focuses on modeling the conditional entropy between frames.
We first show that a simple architecture modeling the entropy between the image latent codes is as competitive as other neural video compression works and video codecs.
We then propose a novel internal learning extension on top of this architecture that brings an additional 10% savings without trading off decoding speed.
arXiv Detail & Related papers (2020-08-20T20:01:59Z) - Modeling Lost Information in Lossy Image Compression [72.69327382643549]
Lossy image compression is one of the most commonly used operators for digital images.
We propose a novel invertible framework called Invertible Lossy Compression (ILC) to largely mitigate the information loss problem.
arXiv Detail & Related papers (2020-06-22T04:04:56Z) - Content Adaptive and Error Propagation Aware Deep Video Compression [110.31693187153084]
We propose a content adaptive and error propagation aware video compression system.
Our method employs a joint training strategy by considering the compression performance of multiple consecutive frames instead of a single frame.
Instead of using the hand-crafted coding modes in the traditional compression systems, we design an online encoder updating scheme in our system.
arXiv Detail & Related papers (2020-03-25T09:04:24Z) - Learning for Video Compression with Hierarchical Quality and Recurrent
Enhancement [164.7489982837475]
We propose a Hierarchical Learned Video Compression (HLVC) method with three hierarchical quality layers and a recurrent enhancement network.
In our HLVC approach, the hierarchical quality benefits the coding efficiency, since the high quality information facilitates the compression and enhancement of low quality frames at encoder and decoder sides.
arXiv Detail & Related papers (2020-03-04T09:31:37Z) - A Unified End-to-End Framework for Efficient Deep Image Compression [35.156677716140635]
We propose a unified framework called Efficient Deep Image Compression (EDIC) based on three new technologies.
Specifically, we design an auto-encoder style network for learning based image compression.
Our EDIC method can also be readily incorporated with the Deep Video Compression (DVC) framework to further improve the video compression performance.
arXiv Detail & Related papers (2020-02-09T14:21:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.