Rethinking Data Augmentation for Image Super-resolution: A Comprehensive Analysis and a New Strategy

• URL: http://arxiv.org/abs/2004.00448v2
• Date: Thu, 23 Apr 2020 08:28:10 GMT
• Title: Rethinking Data Augmentation for Image Super-resolution: A Comprehensive Analysis and a New Strategy
• Authors: Jaejun Yoo, Namhyuk Ahn, Kyung-Ah Sohn
• Abstract summary: We provide a comprehensive analysis of the existing augmentation methods applied to the super-resolution task. We propose CutBlur that cuts a low-resolution patch and pastes it to the corresponding high-resolution image region and vice versa. Our method consistently and significantly improves the performance across various scenarios.
• Score: 21.89072742618842
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Data augmentation is an effective way to improve the performance of deep networks. Unfortunately, current methods are mostly developed for high-level vision tasks (e.g., classification) and few are studied for low-level vision tasks (e.g., image restoration). In this paper, we provide a comprehensive analysis of the existing augmentation methods applied to the super-resolution task. We find that the methods discarding or manipulating the pixels or features too much hamper the image restoration, where the spatial relationship is very important. Based on our analyses, we propose CutBlur that cuts a low-resolution patch and pastes it to the corresponding high-resolution image region and vice versa. The key intuition of CutBlur is to enable a model to learn not only "how" but also "where" to super-resolve an image. By doing so, the model can understand "how much", instead of blindly learning to apply super-resolution to every given pixel. Our method consistently and significantly improves the performance across various scenarios, especially when the model size is big and the data is collected under real-world environments. We also show that our method improves other low-level vision tasks, such as denoising and compression artifact removal.

Related papers

• Research on Image Super-Resolution Reconstruction Mechanism based on Convolutional Neural Network [8.739451985459638]
  Super-resolution algorithms transform one or more sets of low-resolution images captured from the same scene into high-resolution images. The extraction of image features and nonlinear mapping methods in the reconstruction process remain challenging for existing algorithms. The objective is to recover high-quality, high-resolution images from low-resolution images.
  arXiv  Detail & Related papers  (2024-07-18T06:50:39Z)
• On the Effect of Image Resolution on Semantic Segmentation [27.115235051091663]
  We show that a model capable of directly producing high-resolution segmentations can match the performance of more complex systems. Our approach leverages a bottom-up information propagation technique across various scales. We have rigorously tested our method using leading-edge semantic segmentation datasets.
  arXiv  Detail & Related papers  (2024-02-08T04:21:30Z)
• Unsupervised Deep Learning-based Pansharpening with Jointly-Enhanced Spectral and Spatial Fidelity [4.425982186154401]
  We propose a new deep learning-based pansharpening model that fully exploits the potential of this approach. The proposed model features a novel loss function that jointly promotes the spectral and spatial quality of the pansharpened data. Experiments on a large variety of test images, performed in challenging scenarios, demonstrate that the proposed method compares favorably with the state of the art.
  arXiv  Detail & Related papers  (2023-07-26T17:25:28Z)
• Super-Resolving Face Image by Facial Parsing Information [52.1267613768555]
  Face super-resolution is a technology that transforms a low-resolution face image into the corresponding high-resolution one. We build a novel parsing map guided face super-resolution network which extracts the face prior from low-resolution face image. High-resolution features contain more precise spatial information while low-resolution features provide strong contextual information.
  arXiv  Detail & Related papers  (2023-04-06T08:19:03Z)
• Toward Super-Resolution for Appearance-Based Gaze Estimation [4.594159253008448]
  Super-resolution has been shown to improve image quality from a visual perspective. We propose a two-step framework based on SwinIR super-resolution model. Self-supervised learning aims to learn from unlabelled data to reduce the amount of required labeled data for downstream tasks.
  arXiv  Detail & Related papers  (2023-03-17T17:40:32Z)
• Learning Degradation Representations for Image Deblurring [37.80709422920307]
  We propose a framework to learn spatially adaptive degradation representations of blurry images. A novel joint image reblurring and deblurring learning process is presented to improve the expressiveness of degradation representations. Experiments on the GoPro and RealBlur datasets demonstrate that our proposed deblurring framework with the learned degradation representations outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-08-10T09:53:16Z)
• Learning Weighting Map for Bit-Depth Expansion within a Rational Range [64.15915577164894]
  Bit-depth expansion (BDE) is one of the emerging technologies to display high bit-depth (HBD) image from low bit-depth (LBD) source. Existing BDE methods have no unified solution for various BDE situations. We design a bit restoration network (BRNet) to learn a weight for each pixel, which indicates the ratio of the replenished value within a rational range.
  arXiv  Detail & Related papers  (2022-04-26T02:27:39Z)
• Learning Enriched Features for Fast Image Restoration and Enhancement [166.17296369600774]
  This paper presents a holistic goal of maintaining spatially-precise high-resolution representations through the entire network. We learn an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details. Our approach achieves state-of-the-art results for a variety of image processing tasks, including defocus deblurring, image denoising, super-resolution, and image enhancement.
  arXiv  Detail & Related papers  (2022-04-19T17:59:45Z)
• Invertible Image Rescaling [118.2653765756915]
  We develop an Invertible Rescaling Net (IRN) to produce visually-pleasing low-resolution images. We capture the distribution of the lost information using a latent variable following a specified distribution in the downscaling process.
  arXiv  Detail & Related papers  (2020-05-12T09:55:53Z)
• Learning Enriched Features for Real Image Restoration and Enhancement [166.17296369600774]
  convolutional neural networks (CNNs) have achieved dramatic improvements over conventional approaches for image restoration task. We present a novel architecture with the collective goals of maintaining spatially-precise high-resolution representations through the entire network. Our approach learns an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details.
  arXiv  Detail & Related papers  (2020-03-15T11:04:30Z)
• Gated Fusion Network for Degraded Image Super Resolution [78.67168802945069]
  We propose a dual-branch convolutional neural network to extract base features and recovered features separately. By decomposing the feature extraction step into two task-independent streams, the dual-branch model can facilitate the training process.
  arXiv  Detail & Related papers  (2020-03-02T13:28:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.