Leveraging Bitstream Metadata for Fast, Accurate, Generalized Compressed Video Quality Enhancement

• URL: http://arxiv.org/abs/2202.00011v3
• Date: Mon, 30 Oct 2023 13:47:43 GMT
• Title: Leveraging Bitstream Metadata for Fast, Accurate, Generalized Compressed Video Quality Enhancement
• Authors: Max Ehrlich, Jon Barker, Namitha Padmanabhan, Larry Davis, Andrew Tao, Bryan Catanzaro, Abhinav Shrivastava
• Abstract summary: We develop a deep learning architecture capable of restoring detail to compressed videos. We show that this improves restoration accuracy compared to prior compression correction methods. We condition our model on quantization data which is readily available in the bitstream.
• Score: 74.1052624663082
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video compression is a central feature of the modern internet powering technologies from social media to video conferencing. While video compression continues to mature, for many compression settings, quality loss is still noticeable. These settings nevertheless have important applications to the efficient transmission of videos over bandwidth constrained or otherwise unstable connections. In this work, we develop a deep learning architecture capable of restoring detail to compressed videos which leverages the underlying structure and motion information embedded in the video bitstream. We show that this improves restoration accuracy compared to prior compression correction methods and is competitive when compared with recent deep-learning-based video compression methods on rate-distortion while achieving higher throughput. Furthermore, we condition our model on quantization data which is readily available in the bitstream. This allows our single model to handle a variety of different compression quality settings which required an ensemble of models in prior work.

Related papers

• Progressive Growing of Video Tokenizers for Highly Compressed Latent Spaces [20.860632218272094]
  Video tokenizers are essential for latent video diffusion models, converting raw video data into latent spaces for efficient training. We propose an alternative approach to enhance temporal compression. We develop a bootstrapped high-temporal-compression model that progressively trains high-compression blocks atop well-trained lower-compression models.
  arXiv  Detail & Related papers  (2025-01-09T18:55:15Z)
• Large Motion Video Autoencoding with Cross-modal Video VAE [52.13379965800485]
  Video Variational Autoencoder (VAE) is essential for reducing video redundancy and facilitating efficient video generation. Existing Video VAEs have begun to address temporal compression; however, they often suffer from inadequate reconstruction performance. We present a novel and powerful video autoencoder capable of high-fidelity video encoding.
  arXiv  Detail & Related papers  (2024-12-23T18:58:24Z)
• Perceptual Quality Improvement in Videoconferencing using Keyframes-based GAN [28.773037051085318]
  We propose a novel GAN-based method for compression artifacts reduction in videoconferencing. First, we extract multi-scale features from the compressed and reference frames. Then, our architecture combines these features in a progressive manner according to facial landmarks.
  arXiv  Detail & Related papers  (2023-11-07T16:38:23Z)
• Valid Information Guidance Network for Compressed Video Quality Enhancement [10.294638746269298]
  We propose a unique Valid Information Guidance scheme (VIG) to enhance the quality of compressed videos. Our method achieves the state-of-the-art performance of compressed video quality enhancement in terms of accuracy and efficiency.
  arXiv  Detail & Related papers  (2023-02-28T05:43:25Z)
• A Unified Image Preprocessing Framework For Image Compression [5.813935823171752]
  We propose a unified image compression preprocessing framework, called Kuchen, to improve the performance of existing codecs. The framework consists of a hybrid data labeling system along with a learning-based backbone to simulate personalized preprocessing. Results demonstrate that the modern codecs optimized by our unified preprocessing framework constantly improve the efficiency of the state-of-the-art compression.
  arXiv  Detail & Related papers  (2022-08-15T10:41:00Z)
• Learned Video Compression via Heterogeneous Deformable Compensation Network [78.72508633457392]
  We propose a learned video compression framework via heterogeneous deformable compensation strategy (HDCVC) to tackle the problems of unstable compression performance. More specifically, the proposed algorithm extracts features from the two adjacent frames to estimate content-Neighborhood heterogeneous deformable (HetDeform) kernel offsets. Experimental results indicate that HDCVC achieves superior performance than the recent state-of-the-art learned video compression approaches.
  arXiv  Detail & Related papers  (2022-07-11T02:31:31Z)
• COMISR: Compression-Informed Video Super-Resolution [76.94152284740858]
  Most videos on the web or mobile devices are compressed, and the compression can be severe when the bandwidth is limited. We propose a new compression-informed video super-resolution model to restore high-resolution content without introducing artifacts caused by compression.
  arXiv  Detail & Related papers  (2021-05-04T01:24:44Z)
• Content Adaptive and Error Propagation Aware Deep Video Compression [110.31693187153084]
  We propose a content adaptive and error propagation aware video compression system. Our method employs a joint training strategy by considering the compression performance of multiple consecutive frames instead of a single frame. Instead of using the hand-crafted coding modes in the traditional compression systems, we design an online encoder updating scheme in our system.
  arXiv  Detail & Related papers  (2020-03-25T09:04:24Z)
• Learning for Video Compression with Hierarchical Quality and Recurrent Enhancement [164.7489982837475]
  We propose a Hierarchical Learned Video Compression (HLVC) method with three hierarchical quality layers and a recurrent enhancement network. In our HLVC approach, the hierarchical quality benefits the coding efficiency, since the high quality information facilitates the compression and enhancement of low quality frames at encoder and decoder sides.
  arXiv  Detail & Related papers  (2020-03-04T09:31:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.