Both Style and Fog Matter: Cumulative Domain Adaptation for Semantic Foggy Scene Understanding

• URL: http://arxiv.org/abs/2112.00484v1
• Date: Wed, 1 Dec 2021 13:21:20 GMT
• Title: Both Style and Fog Matter: Cumulative Domain Adaptation for Semantic Foggy Scene Understanding
• Authors: Xianzheng Ma, Zhixiang Wang, Yacheng Zhan, Yinqiang Zheng, Zheng Wang, Dengxin Dai, Chia-Wen Lin
• Abstract summary: We propose a new pipeline to cumulatively adapt style, fog and the dual-factor (style and fog) Specifically, we devise a unified framework to disentangle the style factor and the fog factor separately, and then the dual-factor from images in different domains. Our method achieves the state-of-the-art performance on three benchmarks and shows generalization ability in rainy and snowy scenes.
• Score: 63.99301797430936
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Although considerable progress has been made in semantic scene understanding under clear weather, it is still a tough problem under adverse weather conditions, such as dense fog, due to the uncertainty caused by imperfect observations. Besides, difficulties in collecting and labeling foggy images hinder the progress of this field. Considering the success in semantic scene understanding under clear weather, we think it is reasonable to transfer knowledge learned from clear images to the foggy domain. As such, the problem becomes to bridge the domain gap between clear images and foggy images. Unlike previous methods that mainly focus on closing the domain gap caused by fog -- defogging the foggy images or fogging the clear images, we propose to alleviate the domain gap by considering fog influence and style variation simultaneously. The motivation is based on our finding that the style-related gap and the fog-related gap can be divided and closed respectively, by adding an intermediate domain. Thus, we propose a new pipeline to cumulatively adapt style, fog and the dual-factor (style and fog). Specifically, we devise a unified framework to disentangle the style factor and the fog factor separately, and then the dual-factor from images in different domains. Furthermore, we collaborate the disentanglement of three factors with a novel cumulative loss to thoroughly disentangle these three factors. Our method achieves the state-of-the-art performance on three benchmarks and shows generalization ability in rainy and snowy scenes.

Related papers

• Distractors-Immune Representation Learning with Cross-modal Contrastive Regularization for Change Captioning [71.14084801851381]
  Change captioning aims to succinctly describe the semantic change between a pair of similar images. Most existing methods directly capture the difference between them, which risk obtaining error-prone difference features. We propose a distractors-immune representation learning network that correlates the corresponding channels of two image representations.
  arXiv  Detail & Related papers  (2024-07-16T13:00:33Z)
• D2SL: Decouple Defogging and Semantic Learning for Foggy Domain-Adaptive Segmentation [0.8261182037130406]
  We propose a novel training framework, Decouple Defogging and Semantic learning, called D2SL. We introduce a domain-consistent transfer strategy to establish a connection between defogging and segmentation tasks. We design a real fog transfer strategy to improve defogging effects by fully leveraging the fog priors from real foggy images.
  arXiv  Detail & Related papers  (2024-04-07T04:55:58Z)
• Be Yourself: Bounded Attention for Multi-Subject Text-to-Image Generation [60.943159830780154]
  We introduce Bounded Attention, a training-free method for bounding the information flow in the sampling process. We demonstrate that our method empowers the generation of multiple subjects that better align with given prompts and layouts.
  arXiv  Detail & Related papers  (2024-03-25T17:52:07Z)
• Structure Representation Network and Uncertainty Feedback Learning for Dense Non-Uniform Fog Removal [64.77435210892041]
  We introduce a structure-representation network with uncertainty feedback learning. Specifically, we extract the feature representations from a pre-trained Vision Transformer (DINO-ViT) module to recover the background information. To handle the intractability of estimating the atmospheric light colors, we exploit the grayscale version of our input image.
  arXiv  Detail & Related papers  (2022-10-06T17:10:57Z)
• Unsupervised Foggy Scene Understanding via Self Spatial-Temporal Label Diffusion [51.11295961195151]
  We exploit the characteristics of the foggy image sequence of driving scenes to densify the confident pseudo labels. Based on the two discoveries of local spatial similarity and adjacent temporal correspondence of the sequential image data, we propose a novel Target-Domain driven pseudo label Diffusion scheme. Our scheme helps the adaptive model achieve 51.92% and 53.84% mean intersection-over-union (mIoU) on two publicly available natural foggy datasets.
  arXiv  Detail & Related papers  (2022-06-10T05:16:50Z)
• FIFO: Learning Fog-invariant Features for Foggy Scene Segmentation [14.932318540666548]
  We propose a new method for learning semantic segmentation models robust against fog. Its key idea is to consider the fog condition of an image as its style and close the gap between images with different fog conditions. Our method substantially outperforms previous work on three real foggy image datasets.
  arXiv  Detail & Related papers  (2022-04-04T15:33:42Z)
• Unsupervised Deraining: Where Contrastive Learning Meets Self-similarity [0.0]
  We propose a novel non-local contrastive learning (NLCL) method for unsupervised image deraining. The proposed method obtains state-of-the-art performance in real deraining.
  arXiv  Detail & Related papers  (2022-03-22T07:37:08Z)
• Leveraging Scale-Invariance and Uncertainity with Self-Supervised Domain Adaptation for Semantic Segmentation of Foggy Scenes [4.033107207078282]
  FogAdapt is a novel approach for domain adaptation of semantic segmentation for dense foggy scenes. FogAdapt significantly outperforms the current state-of-the-art in semantic segmentation of foggy images.
  arXiv  Detail & Related papers  (2022-01-07T18:29:58Z)
• Non-Homogeneous Haze Removal via Artificial Scene Prior and Bidimensional Graph Reasoning [52.07698484363237]
  We propose a Non-Homogeneous Haze Removal Network (NHRN) via artificial scene prior and bidimensional graph reasoning. Our method achieves superior performance over many state-of-the-art algorithms for both the single image dehazing and hazy image understanding tasks.
  arXiv  Detail & Related papers  (2021-04-05T13:04:44Z)
• Adherent Mist and Raindrop Removal from a Single Image Using Attentive Convolutional Network [1.2891210250935146]
  Temperature difference-induced mist adhered to the glass, such as windshield, camera lens, is often inhomogeneous and obscure. In this work, we newly present a problem of image degradation caused by adherent mist and raindrops. An attentive convolutional network is adopted to visually remove the adherent mist and raindrop from a single image.
  arXiv  Detail & Related papers  (2020-09-03T06:17:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.