Semantically Accurate Super-Resolution Generative Adversarial Networks

• URL: http://arxiv.org/abs/2205.08659v1
• Date: Tue, 17 May 2022 23:05:27 GMT
• Title: Semantically Accurate Super-Resolution Generative Adversarial Networks
• Authors: Tristan Frizza and Donald G. Dansereau and Nagita Mehr Seresht and Michael Bewley
• Abstract summary: We propose a novel architecture and domain-specific feature loss to increase the performance of semantic segmentation. We show the proposed approach improves perceived image quality as well as quantitative segmentation accuracy across all prediction classes. This work demonstrates that jointly considering image-based and task-specific losses can improve the performance of both, and advances the state-of-the-art in semantic-aware super-resolution of aerial imagery.
• Score: 2.0454959820861727
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This work addresses the problems of semantic segmentation and image super-resolution by jointly considering the performance of both in training a Generative Adversarial Network (GAN). We propose a novel architecture and domain-specific feature loss, allowing super-resolution to operate as a pre-processing step to increase the performance of downstream computer vision tasks, specifically semantic segmentation. We demonstrate this approach using Nearmap's aerial imagery dataset which covers hundreds of urban areas at 5-7 cm per pixel resolution. We show the proposed approach improves perceived image quality as well as quantitative segmentation accuracy across all prediction classes, yielding an average accuracy improvement of 11.8% and 108% at 4x and 32x super-resolution, compared with state-of-the art single-network methods. This work demonstrates that jointly considering image-based and task-specific losses can improve the performance of both, and advances the state-of-the-art in semantic-aware super-resolution of aerial imagery.

Related papers

• On the Effect of Image Resolution on Semantic Segmentation [27.115235051091663]
  We show that a model capable of directly producing high-resolution segmentations can match the performance of more complex systems. Our approach leverages a bottom-up information propagation technique across various scales. We have rigorously tested our method using leading-edge semantic segmentation datasets.
  arXiv  Detail & Related papers  (2024-02-08T04:21:30Z)
• Intra-Batch Supervision for Panoptic Segmentation on High-Resolution Images [4.314956204483074]
  Unified panoptic segmentation methods are achieving state-of-the-art results on several datasets. To achieve these results on high-resolution datasets, these methods apply crop-based training. We find that, although crop-based training is advantageous in general, it also has a harmful side-effect. We propose Intra-Batch Supervision (IBS), which improves a network's ability to discriminate between instances.
  arXiv  Detail & Related papers  (2023-04-17T12:48:36Z)
• Hierarchical Similarity Learning for Aliasing Suppression Image Super-Resolution [64.15915577164894]
  A hierarchical image super-resolution network (HSRNet) is proposed to suppress the influence of aliasing. HSRNet achieves better quantitative and visual performance than other works, and remits the aliasing more effectively.
  arXiv  Detail & Related papers  (2022-06-07T14:55:32Z)
• High Quality Segmentation for Ultra High-resolution Images [72.97958314291648]
  We propose the Continuous Refinement Model for the ultra high-resolution segmentation refinement task. Our proposed method is fast and effective on image segmentation refinement.
  arXiv  Detail & Related papers  (2021-11-29T11:53:06Z)
• Spatially-Adaptive Image Restoration using Distortion-Guided Networks [51.89245800461537]
  We present a learning-based solution for restoring images suffering from spatially-varying degradations. We propose SPAIR, a network design that harnesses distortion-localization information and dynamically adjusts to difficult regions in the image.
  arXiv  Detail & Related papers  (2021-08-19T11:02:25Z)
• Analysis and evaluation of Deep Learning based Super-Resolution algorithms to improve performance in Low-Resolution Face Recognition [0.0]
  Super-resolution algorithms may be able to recover the discriminant properties of the subjects involved. This project aimed at evaluating and adapting different deep neural network architectures for the task of face super-resolution. Experiments showed that general super-resolution architectures might enhance face verification performance of deep neural networks trained on high-resolution faces.
  arXiv  Detail & Related papers  (2021-01-19T02:41:57Z)
• Semantic Layout Manipulation with High-Resolution Sparse Attention [106.59650698907953]
  We tackle the problem of semantic image layout manipulation, which aims to manipulate an input image by editing its semantic label map. A core problem of this task is how to transfer visual details from the input images to the new semantic layout while making the resulting image visually realistic. We propose a high-resolution sparse attention module that effectively transfers visual details to new layouts at a resolution up to 512x512.
  arXiv  Detail & Related papers  (2020-12-14T06:50:43Z)
• Interpretable Detail-Fidelity Attention Network for Single Image Super-Resolution [89.1947690981471]
  We propose a purposeful and interpretable detail-fidelity attention network to progressively process smoothes and details in divide-and-conquer manner. Particularly, we propose a Hessian filtering for interpretable feature representation which is high-profile for detail inference. Experiments demonstrate that the proposed methods achieve superior performances over the state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-09-28T08:31:23Z)
• Rethinking Data Augmentation for Image Super-resolution: A Comprehensive Analysis and a New Strategy [21.89072742618842]
  We provide a comprehensive analysis of the existing augmentation methods applied to the super-resolution task. We propose CutBlur that cuts a low-resolution patch and pastes it to the corresponding high-resolution image region and vice versa. Our method consistently and significantly improves the performance across various scenarios.
  arXiv  Detail & Related papers  (2020-04-01T13:49:38Z)
• Gated Fusion Network for Degraded Image Super Resolution [78.67168802945069]
  We propose a dual-branch convolutional neural network to extract base features and recovered features separately. By decomposing the feature extraction step into two task-independent streams, the dual-branch model can facilitate the training process.
  arXiv  Detail & Related papers  (2020-03-02T13:28:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.