Unsupervised Restoration of Weather-affected Images using Deep Gaussian Process-based CycleGAN

• URL: http://arxiv.org/abs/2204.10970v1
• Date: Sat, 23 Apr 2022 01:30:47 GMT
• Title: Unsupervised Restoration of Weather-affected Images using Deep Gaussian Process-based CycleGAN
• Authors: Rajeev Yasarla, Vishwanath A. Sindagi, Vishal M. Patel
• Abstract summary: We describe an approach for supervising deep networks that are based on CycleGAN. We introduce new losses for training CycleGAN that lead to more effective training, resulting in high-quality reconstructions. We demonstrate that the proposed method can be effectively applied to different restoration tasks like de-raining, de-hazing and de-snowing.
• Score: 92.15895515035795
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing approaches for restoring weather-degraded images follow a fully-supervised paradigm and they require paired data for training. However, collecting paired data for weather degradations is extremely challenging, and existing methods end up training on synthetic data. To overcome this issue, we describe an approach for supervising deep networks that are based on CycleGAN, thereby enabling the use of unlabeled real-world data for training. Specifically, we introduce new losses for training CycleGAN that lead to more effective training, resulting in high-quality reconstructions. These new losses are obtained by jointly modeling the latent space embeddings of predicted clean images and original clean images through Deep Gaussian Processes. This enables the CycleGAN architecture to transfer the knowledge from one domain (weather-degraded) to another (clean) more effectively. We demonstrate that the proposed method can be effectively applied to different restoration tasks like de-raining, de-hazing and de-snowing and it outperforms other unsupervised techniques (that leverage weather-based characteristics) by a considerable margin.

Related papers

• Continual All-in-One Adverse Weather Removal with Knowledge Replay on a Unified Network Structure [92.8834309803903]
  In real-world applications, image degeneration caused by adverse weather is always complex and changes with different weather conditions from days and seasons. We develop a novel continual learning framework with effective knowledge replay (KR) on a unified network structure. It considers the characteristics of the image restoration task with multiple degenerations in continual learning, and the knowledge for different degenerations can be shared and accumulated.
  arXiv  Detail & Related papers  (2024-03-12T03:50:57Z)
• Data-efficient Event Camera Pre-training via Disentangled Masked Modeling [20.987277885575963]
  We present a new data-supervised voxel-based self-supervised learning method for event cameras. Our method overcomes the limitations of previous methods, which either sacrifice temporal information or directly employ paired image data. It exhibits excellent generalization performance and demonstrates significant improvements across various tasks with fewer parameters and lower computational costs.
  arXiv  Detail & Related papers  (2024-03-01T10:02:25Z)
• WeatherProof: A Paired-Dataset Approach to Semantic Segmentation in Adverse Weather [9.619700283574533]
  We introduce a general paired-training method that leads to improved performance on images in adverse weather conditions. We create the first semantic segmentation dataset with accurate clear and adverse weather image pairs. We find that training on these paired clear and adverse weather frames which share an underlying scene results in improved performance on adverse weather data.
  arXiv  Detail & Related papers  (2023-12-15T04:57:54Z)
• DGNet: Dynamic Gradient-Guided Network for Water-Related Optics Image Enhancement [77.0360085530701]
  Underwater image enhancement (UIE) is a challenging task due to the complex degradation caused by underwater environments. Previous methods often idealize the degradation process, and neglect the impact of medium noise and object motion on the distribution of image features. Our approach utilizes predicted images to dynamically update pseudo-labels, adding a dynamic gradient to optimize the network's gradient space.
  arXiv  Detail & Related papers  (2023-12-12T06:07:21Z)
• Accelerating Multiframe Blind Deconvolution via Deep Learning [0.0]
  Ground-based solar image restoration is a computationally expensive procedure. We propose a new method to accelerate the restoration based on algorithm unrolling. We show that both methods significantly reduce the restoration time compared to the standard optimization procedure.
  arXiv  Detail & Related papers  (2023-06-21T07:53:00Z)
• Exploring the Application of Large-scale Pre-trained Models on Adverse Weather Removal [97.53040662243768]
  We propose a CLIP embedding module to make the network handle different weather conditions adaptively. This module integrates the sample specific weather prior extracted by CLIP image encoder together with the distribution specific information learned by a set of parameters.
  arXiv  Detail & Related papers  (2023-06-15T10:06:13Z)
• A comparison of different atmospheric turbulence simulation methods for image restoration [64.24948495708337]
  Atmospheric turbulence deteriorates the quality of images captured by long-range imaging systems. Various deep learning-based atmospheric turbulence mitigation methods have been proposed in the literature. We systematically evaluate the effectiveness of various turbulence simulation methods on image restoration.
  arXiv  Detail & Related papers  (2022-04-19T16:21:36Z)
• ART-SS: An Adaptive Rejection Technique for Semi-Supervised restoration for adverse weather-affected images [24.03416814412226]
  We study the effect of unlabeled data on the performance of an SSR method. We develop a technique that rejects the unlabeled images that degrade the performance.
  arXiv  Detail & Related papers  (2022-03-17T12:00:31Z)
• Understanding Training-Data Leakage from Gradients in Neural Networks for Image Classification [11.272188531829016]
  In many applications, we need to protect the training data from being leaked due to IP or privacy concerns. Recent works have demonstrated that it is possible to reconstruct the training data from gradients for an image-classification model when its architecture is known. We formulate the problem of training data reconstruction as solving an optimisation problem iteratively for each layer. We are able to attribute the potential leakage of the training data in a deep network to its architecture.
  arXiv  Detail & Related papers  (2021-11-19T12:14:43Z)
• Syn2Real Transfer Learning for Image Deraining using Gaussian Processes [92.15895515035795]
  CNN-based methods for image deraining have achieved excellent performance in terms of reconstruction error as well as visual quality. Due to challenges in obtaining real world fully-labeled image deraining datasets, existing methods are trained only on synthetically generated data. We propose a Gaussian Process-based semi-supervised learning framework which enables the network in learning to derain using synthetic dataset.
  arXiv  Detail & Related papers  (2020-06-10T00:33:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.