Expanding Synthetic Real-World Degradations for Blind Video Super Resolution

• URL: http://arxiv.org/abs/2305.02660v1
• Date: Thu, 4 May 2023 08:58:31 GMT
• Title: Expanding Synthetic Real-World Degradations for Blind Video Super Resolution
• Authors: Mehran Jeelani, Sadbhawna, Noshaba Cheema, Klaus Illgner-Fehns, Philipp Slusallek, and Sunil Jaiswal
• Abstract summary: Video super-resolution (VSR) techniques have drastically improved over the last few years and shown impressive performance on synthetic data. However, their performance on real-world video data suffers because of the complexity of real-world degradations and misaligned video frames. In this paper, we propose real-world degradations on synthetic training datasets.
• Score: 3.474523163017713
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Video super-resolution (VSR) techniques, especially deep-learning-based algorithms, have drastically improved over the last few years and shown impressive performance on synthetic data. However, their performance on real-world video data suffers because of the complexity of real-world degradations and misaligned video frames. Since obtaining a synthetic dataset consisting of low-resolution (LR) and high-resolution (HR) frames are easier than obtaining real-world LR and HR images, in this paper, we propose synthesizing real-world degradations on synthetic training datasets. The proposed synthetic real-world degradations (SRWD) include a combination of the blur, noise, downsampling, pixel binning, and image and video compression artifacts. We then propose using a random shuffling-based strategy to simulate these degradations on the training datasets and train a single end-to-end deep neural network (DNN) on the proposed larger variation of realistic synthesized training data. Our quantitative and qualitative comparative analysis shows that the proposed training strategy using diverse realistic degradations improves the performance by 7.1 % in terms of NRQM compared to RealBasicVSR and by 3.34 % compared to BSRGAN on the VideoLQ dataset. We also introduce a new dataset that contains high-resolution real-world videos that can serve as a common ground for bench-marking.

Related papers

• Enhanced Super-Resolution Training via Mimicked Alignment for Real-World Scenes [51.92255321684027]
  We propose a novel plug-and-play module designed to mitigate misalignment issues by aligning LR inputs with HR images during training. Specifically, our approach involves mimicking a novel LR sample that aligns with HR while preserving the characteristics of the original LR samples. We comprehensively evaluate our method on synthetic and real-world datasets, demonstrating its effectiveness across a spectrum of SR models.
  arXiv  Detail & Related papers  (2024-10-07T18:18:54Z)
• Towards Realistic Data Generation for Real-World Super-Resolution [58.88039242455039]
  RealDGen is an unsupervised learning data generation framework designed for real-world super-resolution. We develop content and degradation extraction strategies, which are integrated into a novel content-degradation decoupled diffusion model. Experiments demonstrate that RealDGen excels in generating large-scale, high-quality paired data that mirrors real-world degradations.
  arXiv  Detail & Related papers  (2024-06-11T13:34:57Z)
• VCISR: Blind Single Image Super-Resolution with Video Compression Synthetic Data [18.877077302923713]
  We present a video compression-based degradation model to synthesize low-resolution image data in the blind SISR task. Our proposed image synthesizing method is widely applicable to existing image datasets. By introducing video coding artifacts to SISR degradation models, neural networks can super-resolve images with the ability to restore video compression degradations.
  arXiv  Detail & Related papers  (2023-11-02T05:24:19Z)
• Benchmark Dataset and Effective Inter-Frame Alignment for Real-World Video Super-Resolution [65.20905703823965]
  Video super-resolution (VSR) aiming to reconstruct a high-resolution (HR) video from its low-resolution (LR) counterpart has made tremendous progress in recent years. It remains challenging to deploy existing VSR methods to real-world data with complex degradations. EAVSR takes the proposed multi-layer adaptive spatial transform network (MultiAdaSTN) to refine the offsets provided by the pre-trained optical flow estimation network.
  arXiv  Detail & Related papers  (2022-12-10T17:41:46Z)
• Towards Real World HDRTV Reconstruction: A Data Synthesis-based Approach [48.1492764654516]
  Existing deep learning based HDRTV reconstruction methods assume one kind of tone mapping operators (TMOs) as the procedure to synthesize SDRTV-TV pairs for supervised training. In this paper, we argue that, although traditional TMOs exploit efficient dynamic range compression priors, they have several drawbacks on modeling the degradation information over-preservation, color bias and possible artifacts. We propose a learning-based data synthesis approach to learn the properties of real-world SDRTVs by integrating several tone mapping priors into both network structures and loss functions.
  arXiv  Detail & Related papers  (2022-11-06T08:27:37Z)
• Towards Real-World Video Deblurring by Exploring Blur Formation Process [53.91239555063343]
  In recent years, deep learning-based approaches have achieved promising success on video deblurring task. The models trained on existing synthetic datasets still suffer from generalization problems over real-world blurry scenarios. We propose a novel realistic blur synthesis pipeline termed RAW-Blur by leveraging blur formation cues.
  arXiv  Detail & Related papers  (2022-08-28T09:24:52Z)
• Investigating Tradeoffs in Real-World Video Super-Resolution [90.81396836308085]
  Real-world video super-resolution (VSR) models are often trained with diverse degradations to improve generalizability. To alleviate the first tradeoff, we propose a degradation scheme that reduces up to 40% of training time without sacrificing performance. To facilitate fair comparisons, we propose the new VideoLQ dataset, which contains a large variety of real-world low-quality video sequences.
  arXiv  Detail & Related papers  (2021-11-24T18:58:21Z)
• Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data [17.529045507657944]
  We extend the powerful ESRGAN to a practical restoration application (namely, Real-ESRGAN) A high-order degradation modeling process is introduced to better simulate complex real-world degradations. We also consider the common ringing and overshoot artifacts in the synthesis process.
  arXiv  Detail & Related papers  (2021-07-22T17:43:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.