Leveraging Bitstream Metadata for Fast, Accurate, Generalized Compressed Video Quality Enhancement

• URL: http://arxiv.org/abs/2202.00011v3
• Date: Mon, 30 Oct 2023 13:47:43 GMT
• Title: Leveraging Bitstream Metadata for Fast, Accurate, Generalized Compressed Video Quality Enhancement
• Authors: Max Ehrlich, Jon Barker, Namitha Padmanabhan, Larry Davis, Andrew Tao, Bryan Catanzaro, Abhinav Shrivastava
• Abstract summary: We develop a deep learning architecture capable of restoring detail to compressed videos. We show that this improves restoration accuracy compared to prior compression correction methods. We condition our model on quantization data which is readily available in the bitstream.
• Score: 74.1052624663082
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video compression is a central feature of the modern internet powering technologies from social media to video conferencing. While video compression continues to mature, for many compression settings, quality loss is still noticeable. These settings nevertheless have important applications to the efficient transmission of videos over bandwidth constrained or otherwise unstable connections. In this work, we develop a deep learning architecture capable of restoring detail to compressed videos which leverages the underlying structure and motion information embedded in the video bitstream. We show that this improves restoration accuracy compared to prior compression correction methods and is competitive when compared with recent deep-learning-based video compression methods on rate-distortion while achieving higher throughput. Furthermore, we condition our model on quantization data which is readily available in the bitstream. This allows our single model to handle a variety of different compression quality settings which required an ensemble of models in prior work.

Related papers

• Plug-and-Play Versatile Compressed Video Enhancement [57.62582951699999]
  Video compression effectively reduces the size of files, making it possible for real-time cloud computing. However, it comes at the cost of visual quality, challenges the robustness of downstream vision models. We present a versatile-aware enhancement framework that adaptively enhance videos under different compression settings.
  arXiv  Detail & Related papers  (2025-04-21T18:39:31Z)
• Embedding Compression Distortion in Video Coding for Machines [67.97469042910855]
  Currently, video transmission serves not only the Human Visual System (HVS) for viewing but also machine perception for analysis. We propose a Compression Distortion Embedding (CDRE) framework, which extracts machine-perception-related distortion representation and embeds it into downstream models. Our framework can effectively boost the rate-task performance of existing codecs with minimal overhead in terms of execution time, and number of parameters.
  arXiv  Detail & Related papers  (2025-03-27T13:01:53Z)
• Progressive Growing of Video Tokenizers for Highly Compressed Latent Spaces [20.860632218272094]
  Video tokenizers are essential for latent video diffusion models, converting raw video data into latent spaces for efficient training. We propose an alternative approach to enhance temporal compression. We develop a bootstrapped high-temporal-compression model that progressively trains high-compression blocks atop well-trained lower-compression models.
  arXiv  Detail & Related papers  (2025-01-09T18:55:15Z)
• Large Motion Video Autoencoding with Cross-modal Video VAE [52.13379965800485]
  Video Variational Autoencoder (VAE) is essential for reducing video redundancy and facilitating efficient video generation. Existing Video VAEs have begun to address temporal compression; however, they often suffer from inadequate reconstruction performance. We present a novel and powerful video autoencoder capable of high-fidelity video encoding.
  arXiv  Detail & Related papers  (2024-12-23T18:58:24Z)
• Blurry Video Compression: A Trade-off between Visual Enhancement and Data Compression [65.8148169700705]
  Existing video compression (VC) methods primarily aim to reduce the spatial and temporal redundancies between consecutive frames in a video. Previous works have achieved remarkable results on videos acquired under specific settings such as instant (known) exposure time and shutter speed. In this work, we tackle the VC problem in a general scenario where a given video can be blurry due to predefined camera settings or dynamics in the scene.
  arXiv  Detail & Related papers  (2023-11-08T02:17:54Z)
• Perceptual Quality Improvement in Videoconferencing using Keyframes-based GAN [28.773037051085318]
  We propose a novel GAN-based method for compression artifacts reduction in videoconferencing. First, we extract multi-scale features from the compressed and reference frames. Then, our architecture combines these features in a progressive manner according to facial landmarks.
  arXiv  Detail & Related papers  (2023-11-07T16:38:23Z)
• Valid Information Guidance Network for Compressed Video Quality Enhancement [10.294638746269298]
  We propose a unique Valid Information Guidance scheme (VIG) to enhance the quality of compressed videos. Our method achieves the state-of-the-art performance of compressed video quality enhancement in terms of accuracy and efficiency.
  arXiv  Detail & Related papers  (2023-02-28T05:43:25Z)
• A Unified Image Preprocessing Framework For Image Compression [5.813935823171752]
  We propose a unified image compression preprocessing framework, called Kuchen, to improve the performance of existing codecs. The framework consists of a hybrid data labeling system along with a learning-based backbone to simulate personalized preprocessing. Results demonstrate that the modern codecs optimized by our unified preprocessing framework constantly improve the efficiency of the state-of-the-art compression.
  arXiv  Detail & Related papers  (2022-08-15T10:41:00Z)
• Learned Video Compression via Heterogeneous Deformable Compensation Network [78.72508633457392]
  We propose a learned video compression framework via heterogeneous deformable compensation strategy (HDCVC) to tackle the problems of unstable compression performance. More specifically, the proposed algorithm extracts features from the two adjacent frames to estimate content-Neighborhood heterogeneous deformable (HetDeform) kernel offsets. Experimental results indicate that HDCVC achieves superior performance than the recent state-of-the-art learned video compression approaches.
  arXiv  Detail & Related papers  (2022-07-11T02:31:31Z)
• COMISR: Compression-Informed Video Super-Resolution [76.94152284740858]
  Most videos on the web or mobile devices are compressed, and the compression can be severe when the bandwidth is limited. We propose a new compression-informed video super-resolution model to restore high-resolution content without introducing artifacts caused by compression.
  arXiv  Detail & Related papers  (2021-05-04T01:24:44Z)
• Content Adaptive and Error Propagation Aware Deep Video Compression [110.31693187153084]
  We propose a content adaptive and error propagation aware video compression system. Our method employs a joint training strategy by considering the compression performance of multiple consecutive frames instead of a single frame. Instead of using the hand-crafted coding modes in the traditional compression systems, we design an online encoder updating scheme in our system.
  arXiv  Detail & Related papers  (2020-03-25T09:04:24Z)
• Learning for Video Compression with Hierarchical Quality and Recurrent Enhancement [164.7489982837475]
  We propose a Hierarchical Learned Video Compression (HLVC) method with three hierarchical quality layers and a recurrent enhancement network. In our HLVC approach, the hierarchical quality benefits the coding efficiency, since the high quality information facilitates the compression and enhancement of low quality frames at encoder and decoder sides.
  arXiv  Detail & Related papers  (2020-03-04T09:31:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.