Leveraging Scale-Invariance and Uncertainity with Self-Supervised Domain Adaptation for Semantic Segmentation of Foggy Scenes

• URL: http://arxiv.org/abs/2201.02588v1
• Date: Fri, 7 Jan 2022 18:29:58 GMT
• Title: Leveraging Scale-Invariance and Uncertainity with Self-Supervised Domain Adaptation for Semantic Segmentation of Foggy Scenes
• Authors: Javed Iqbal, Rehan Hafiz, Mohsen Ali
• Abstract summary: FogAdapt is a novel approach for domain adaptation of semantic segmentation for dense foggy scenes. FogAdapt significantly outperforms the current state-of-the-art in semantic segmentation of foggy images.
• Score: 4.033107207078282
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This paper presents FogAdapt, a novel approach for domain adaptation of semantic segmentation for dense foggy scenes. Although significant research has been directed to reduce the domain shift in semantic segmentation, adaptation to scenes with adverse weather conditions remains an open question. Large variations in the visibility of the scene due to weather conditions, such as fog, smog, and haze, exacerbate the domain shift, thus making unsupervised adaptation in such scenarios challenging. We propose a self-entropy and multi-scale information augmented self-supervised domain adaptation method (FogAdapt) to minimize the domain shift in foggy scenes segmentation. Supported by the empirical evidence that an increase in fog density results in high self-entropy for segmentation probabilities, we introduce a self-entropy based loss function to guide the adaptation method. Furthermore, inferences obtained at different image scales are combined and weighted by the uncertainty to generate scale-invariant pseudo-labels for the target domain. These scale-invariant pseudo-labels are robust to visibility and scale variations. We evaluate the proposed model on real clear-weather scenes to real foggy scenes adaptation and synthetic non-foggy images to real foggy scenes adaptation scenarios. Our experiments demonstrate that FogAdapt significantly outperforms the current state-of-the-art in semantic segmentation of foggy images. Specifically, by considering the standard settings compared to state-of-the-art (SOTA) methods, FogAdapt gains 3.8% on Foggy Zurich, 6.0% on Foggy Driving-dense, and 3.6% on Foggy Driving in mIoU when adapted from Cityscapes to Foggy Zurich.

Related papers

• Data Augmentation via Latent Diffusion for Saliency Prediction [67.88936624546076]
  Saliency prediction models are constrained by the limited diversity and quantity of labeled data. We propose a novel data augmentation method for deep saliency prediction that edits natural images while preserving the complexity and variability of real-world scenes.
  arXiv  Detail & Related papers  (2024-09-11T14:36:24Z)
• D2SL: Decouple Defogging and Semantic Learning for Foggy Domain-Adaptive Segmentation [0.8261182037130406]
  We propose a novel training framework, Decouple Defogging and Semantic learning, called D2SL. We introduce a domain-consistent transfer strategy to establish a connection between defogging and segmentation tasks. We design a real fog transfer strategy to improve defogging effects by fully leveraging the fog priors from real foggy images.
  arXiv  Detail & Related papers  (2024-04-07T04:55:58Z)
• Cameras as Rays: Pose Estimation via Ray Diffusion [54.098613859015856]
  Estimating camera poses is a fundamental task for 3D reconstruction and remains challenging given sparsely sampled views. We propose a distributed representation of camera pose that treats a camera as a bundle of rays. Our proposed methods, both regression- and diffusion-based, demonstrate state-of-the-art performance on camera pose estimation on CO3D.
  arXiv  Detail & Related papers  (2024-02-22T18:59:56Z)
• Domain Adaptation based Object Detection for Autonomous Driving in Foggy and Rainy Weather [44.711384869027775]
  Due to the domain gap, a detection model trained under clear weather may not perform well in foggy and rainy conditions. To bridge the domain gap and improve the performance of object detection in foggy and rainy weather, this paper presents a novel framework for domain-adaptive object detection.
  arXiv  Detail & Related papers  (2023-07-18T23:06:47Z)
• Domain Adaptive Object Detection for Autonomous Driving under Foggy Weather [25.964194141706923]
  This paper proposes a novel domain adaptive object detection framework for autonomous driving under foggy weather. Our method leverages both image-level and object-level adaptation to diminish the domain discrepancy in image style and object appearance. Experimental results on public benchmarks show the effectiveness and accuracy of the proposed method.
  arXiv  Detail & Related papers  (2022-10-27T05:09:10Z)
• Unsupervised Foggy Scene Understanding via Self Spatial-Temporal Label Diffusion [51.11295961195151]
  We exploit the characteristics of the foggy image sequence of driving scenes to densify the confident pseudo labels. Based on the two discoveries of local spatial similarity and adjacent temporal correspondence of the sequential image data, we propose a novel Target-Domain driven pseudo label Diffusion scheme. Our scheme helps the adaptive model achieve 51.92% and 53.84% mean intersection-over-union (mIoU) on two publicly available natural foggy datasets.
  arXiv  Detail & Related papers  (2022-06-10T05:16:50Z)
• FIFO: Learning Fog-invariant Features for Foggy Scene Segmentation [14.932318540666548]
  We propose a new method for learning semantic segmentation models robust against fog. Its key idea is to consider the fog condition of an image as its style and close the gap between images with different fog conditions. Our method substantially outperforms previous work on three real foggy image datasets.
  arXiv  Detail & Related papers  (2022-04-04T15:33:42Z)
• Both Style and Fog Matter: Cumulative Domain Adaptation for Semantic Foggy Scene Understanding [63.99301797430936]
  We propose a new pipeline to cumulatively adapt style, fog and the dual-factor (style and fog) Specifically, we devise a unified framework to disentangle the style factor and the fog factor separately, and then the dual-factor from images in different domains. Our method achieves the state-of-the-art performance on three benchmarks and shows generalization ability in rainy and snowy scenes.
  arXiv  Detail & Related papers  (2021-12-01T13:21:20Z)
• PIT: Position-Invariant Transform for Cross-FoV Domain Adaptation [53.428312630479816]
  We observe that the Field of View (FoV) gap induces noticeable instance appearance differences between the source and target domains. Motivated by the observations, we propose the textbfPosition-Invariant Transform (PIT) to better align images in different domains.
  arXiv  Detail & Related papers  (2021-08-16T15:16:47Z)
• Domain-invariant Similarity Activation Map Contrastive Learning for Retrieval-based Long-term Visual Localization [30.203072945001136]
  In this work, a general architecture is first formulated probabilistically to extract domain invariant feature through multi-domain image translation. And then a novel gradient-weighted similarity activation mapping loss (Grad-SAM) is incorporated for finer localization with high accuracy. Extensive experiments have been conducted to validate the effectiveness of the proposed approach on the CMUSeasons dataset. Our performance is on par with or even outperforms the state-of-the-art image-based localization baselines in medium or high precision.
  arXiv  Detail & Related papers  (2020-09-16T14:43:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.