Efficient Video Compression via Content-Adaptive Super-Resolution
- URL: http://arxiv.org/abs/2104.02322v1
- Date: Tue, 6 Apr 2021 07:01:06 GMT
- Title: Efficient Video Compression via Content-Adaptive Super-Resolution
- Authors: Mehrdad Khani, Vibhaalakshmi Sivaraman, Mohammad Alizadeh
- Abstract summary: Video compression is a critical component of Internet video delivery.
Recent work has shown that deep learning techniques can rival or outperform human algorithms.
This paper presents a new approach that augments a recent deep learning-based video compression scheme.
- Score: 11.6624528293976
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Video compression is a critical component of Internet video delivery. Recent
work has shown that deep learning techniques can rival or outperform
human-designed algorithms, but these methods are significantly less compute and
power-efficient than existing codecs. This paper presents a new approach that
augments existing codecs with a small, content-adaptive super-resolution model
that significantly boosts video quality. Our method, SRVC, encodes video into
two bitstreams: (i) a content stream, produced by compressing downsampled
low-resolution video with the existing codec, (ii) a model stream, which
encodes periodic updates to a lightweight super-resolution neural network
customized for short segments of the video. SRVC decodes the video by passing
the decompressed low-resolution video frames through the (time-varying)
super-resolution model to reconstruct high-resolution video frames. Our results
show that to achieve the same PSNR, SRVC requires 16% of the bits-per-pixel of
H.265 in slow mode, and 2% of the bits-per-pixel of DVC, a recent deep
learning-based video compression scheme. SRVC runs at 90 frames per second on a
NVIDIA V100 GPU.
Related papers
- Fast Encoding and Decoding for Implicit Video Representation [88.43612845776265]
We introduce NeRV-Enc, a transformer-based hyper-network for fast encoding; and NeRV-Dec, a parallel decoder for efficient video loading.
NeRV-Enc achieves an impressive speed-up of $mathbf104times$ by eliminating gradient-based optimization.
NeRV-Dec simplifies video decoding, outperforming conventional codecs with a loading speed $mathbf11times$ faster.
arXiv Detail & Related papers (2024-09-28T18:21:52Z) - When Video Coding Meets Multimodal Large Language Models: A Unified Paradigm for Video Coding [112.44822009714461]
Cross-Modality Video Coding (CMVC) is a pioneering approach to explore multimodality representation and video generative models in video coding.
During decoding, previously encoded components and video generation models are leveraged to create multiple encoding-decoding modes.
Experiments indicate that TT2V achieves effective semantic reconstruction, while IT2V exhibits competitive perceptual consistency.
arXiv Detail & Related papers (2024-08-15T11:36:18Z) - Accelerating Learned Video Compression via Low-Resolution Representation Learning [18.399027308582596]
We introduce an efficiency-optimized framework for learned video compression that focuses on low-resolution representation learning.
Our method achieves performance levels on par with the low-decay P configuration of the H.266 reference software VTM.
arXiv Detail & Related papers (2024-07-23T12:02:57Z) - NU-Class Net: A Novel Approach for Video Quality Enhancement [1.7763979745248648]
This paper introduces NU-Class Net, an innovative deep-learning model designed to mitigate compression artifacts stemming from lossy compression codecs.
By employing the NU-Class Net, the video encoder within the video-capturing node can reduce output quality, thereby generating low-bit-rate videos.
Experimental results affirm the efficacy of the proposed model in enhancing the perceptible quality of videos, especially those streamed at low bit rates.
arXiv Detail & Related papers (2024-01-02T11:46:42Z) - Video Compression with Arbitrary Rescaling Network [8.489428003916622]
We propose a rate-guided arbitrary rescaling network (RARN) for video resizing before encoding.
The lightweight RARN structure can process FHD (1080p) content at real-time speed (91 FPS) and obtain a considerable rate reduction.
arXiv Detail & Related papers (2023-06-07T07:15:18Z) - Block Modulating Video Compression: An Ultra Low Complexity Image Compression Encoder for Resource Limited Platforms [35.76050232152349]
An ultra low-cost image Modulating Video Compression (BMVC) is proposed to be implemented on mobile platforms with low consumption of power and computation resources.
Two types of BMVC decoders, implemented by deep neural networks, are presented.
arXiv Detail & Related papers (2022-05-07T16:20:09Z) - Leveraging Bitstream Metadata for Fast, Accurate, Generalized Compressed
Video Quality Enhancement [74.1052624663082]
We develop a deep learning architecture capable of restoring detail to compressed videos.
We show that this improves restoration accuracy compared to prior compression correction methods.
We condition our model on quantization data which is readily available in the bitstream.
arXiv Detail & Related papers (2022-01-31T18:56:04Z) - COMISR: Compression-Informed Video Super-Resolution [76.94152284740858]
Most videos on the web or mobile devices are compressed, and the compression can be severe when the bandwidth is limited.
We propose a new compression-informed video super-resolution model to restore high-resolution content without introducing artifacts caused by compression.
arXiv Detail & Related papers (2021-05-04T01:24:44Z) - Learning to Compress Videos without Computing Motion [39.46212197928986]
We propose a new deep learning video compression architecture that does not require motion estimation.
Our framework exploits the regularities inherent to video motion, which we capture by using displaced frame differences as video representations.
Our experiments show that our compression model, which we call the MOtionless VIdeo Codec (MOVI-Codec), learns how to efficiently compress videos without computing motion.
arXiv Detail & Related papers (2020-09-29T15:49:25Z) - Conditional Entropy Coding for Efficient Video Compression [82.35389813794372]
We propose a very simple and efficient video compression framework that only focuses on modeling the conditional entropy between frames.
We first show that a simple architecture modeling the entropy between the image latent codes is as competitive as other neural video compression works and video codecs.
We then propose a novel internal learning extension on top of this architecture that brings an additional 10% savings without trading off decoding speed.
arXiv Detail & Related papers (2020-08-20T20:01:59Z) - Learning for Video Compression with Hierarchical Quality and Recurrent
Enhancement [164.7489982837475]
We propose a Hierarchical Learned Video Compression (HLVC) method with three hierarchical quality layers and a recurrent enhancement network.
In our HLVC approach, the hierarchical quality benefits the coding efficiency, since the high quality information facilitates the compression and enhancement of low quality frames at encoder and decoder sides.
arXiv Detail & Related papers (2020-03-04T09:31:37Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.