SR+Codec: a Benchmark of Super-Resolution for Video Compression Bitrate Reduction

• URL: http://arxiv.org/abs/2305.04844v2
• Date: Tue, 20 Aug 2024 15:57:00 GMT
• Title: SR+Codec: a Benchmark of Super-Resolution for Video Compression Bitrate Reduction
• Authors: Evgeney Bogatyrev, Ivan Molodetskikh, Dmitriy Vatolin,
• Abstract summary: We developed a benchmark to analyze Super-Resolution's capacity to upscale compressed videos. Our dataset employed video codecs based on five widely-used compression standards. We found that some SR models, combined with compression, allow us to reduce the video without significant loss of quality.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In recent years, there has been significant interest in Super-Resolution (SR), which focuses on generating a high-resolution image from a low-resolution input. Deep learning-based methods for super-resolution have been particularly popular and have shown impressive results on various benchmarks. However, research indicates that these methods may not perform as well on strongly compressed videos. We developed a super-resolution benchmark to analyze SR's capacity to upscale compressed videos. Our dataset employed video codecs based on five widely-used compression standards: H.264, H.265, H.266, AV1, and AVS3. We assessed 19 popular SR models using our benchmark and evaluated their ability to restore details and their susceptibility to compression artifacts. To get an accurate perceptual ranking of SR models, we conducted a crowd-sourced side-by-side comparison of their outputs. We found that some SR models, combined with compression, allow us to reduce the video bitrate without significant loss of quality. We also compared a range of image and video quality metrics with subjective scores to evaluate their accuracy on super-resolved compressed videos. The benchmark is publicly available at https://videoprocessing.ai/benchmarks/super-resolution-for-video-compression.html

Related papers

• Video Compression with Arbitrary Rescaling Network [8.489428003916622]
  We propose a rate-guided arbitrary rescaling network (RARN) for video resizing before encoding. The lightweight RARN structure can process FHD (1080p) content at real-time speed (91 FPS) and obtain a considerable rate reduction.
  arXiv  Detail & Related papers  (2023-06-07T07:15:18Z)
• SuperTran: Reference Based Video Transformer for Enhancing Low Bitrate Streams in Real Time [0.6308539010172309]
  This work focuses on low video streaming scenarios (e.g. 50 - 200Kbps) where the video quality is severely compromised. We present a family of novel deep generative models for enhancing perceptual video quality of such streams by performing super-resolution while also removing compression artifacts. Our model, which we call SuperTran, consumes as input a single high-quality, high-resolution reference images in addition to the low-quality, low-resolution video stream.
  arXiv  Detail & Related papers  (2022-11-22T22:03:11Z)
• Video compression dataset and benchmark of learning-based video-quality metrics [55.41644538483948]
  We present a new benchmark for video-quality metrics that evaluates video compression. It is based on a new dataset consisting of about 2,500 streams encoded using different standards. Subjective scores were collected using crowdsourced pairwise comparisons.
  arXiv  Detail & Related papers  (2022-11-22T09:22:28Z)
• Compressed Vision for Efficient Video Understanding [83.97689018324732]
  We propose a framework enabling research on hour-long videos with the same hardware that can now process second-long videos. We replace standard video compression, e.g. JPEG, with neural compression and show that we can directly feed compressed videos as inputs to regular video networks.
  arXiv  Detail & Related papers  (2022-10-06T15:35:49Z)
• Learning Spatiotemporal Frequency-Transformer for Compressed Video Super-Resolution [38.00182505384986]
  We propose a novel Frequency-Transformer for compressed video super-resolution (FTVSR) First, we divide a video frame into patches, and transform each patch into DCT spectral maps in which each channel represents a frequency band. Second, we study different self-attention schemes, and discover that a divided attention which conducts a joint space-frequency attention before applying temporal attention on each frequency band, leads to the best video enhancement quality.
  arXiv  Detail & Related papers  (2022-08-05T07:02:30Z)
• VideoINR: Learning Video Implicit Neural Representation for Continuous Space-Time Super-Resolution [75.79379734567604]
  We show that Video Implicit Neural Representation (VideoINR) can be decoded to videos of arbitrary spatial resolution and frame rate. We show that VideoINR achieves competitive performances with state-of-the-art STVSR methods on common up-sampling scales.
  arXiv  Detail & Related papers  (2022-06-09T17:45:49Z)
• Leveraging Bitstream Metadata for Fast, Accurate, Generalized Compressed Video Quality Enhancement [74.1052624663082]
  We develop a deep learning architecture capable of restoring detail to compressed videos. We show that this improves restoration accuracy compared to prior compression correction methods. We condition our model on quantization data which is readily available in the bitstream.
  arXiv  Detail & Related papers  (2022-01-31T18:56:04Z)
• Objective video quality metrics application to video codecs comparisons: choosing the best for subjective quality estimation [101.18253437732933]
  Quality assessment plays a key role in creating and comparing video compression algorithms. For comparison, we used a set of videos encoded with video codecs of different standards, and visual quality scores collected for the resulting set of streams since 2018 until 2021.
  arXiv  Detail & Related papers  (2021-07-21T17:18:11Z)
• COMISR: Compression-Informed Video Super-Resolution [76.94152284740858]
  Most videos on the web or mobile devices are compressed, and the compression can be severe when the bandwidth is limited. We propose a new compression-informed video super-resolution model to restore high-resolution content without introducing artifacts caused by compression.
  arXiv  Detail & Related papers  (2021-05-04T01:24:44Z)
• Efficient Video Compression via Content-Adaptive Super-Resolution [11.6624528293976]
  Video compression is a critical component of Internet video delivery. Recent work has shown that deep learning techniques can rival or outperform human algorithms. This paper presents a new approach that augments a recent deep learning-based video compression scheme.
  arXiv  Detail & Related papers  (2021-04-06T07:01:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.