LoopDA: Constructing Self-loops to Adapt Nighttime Semantic Segmentation

• URL: http://arxiv.org/abs/2211.11870v1
• Date: Mon, 21 Nov 2022 21:46:05 GMT
• Title: LoopDA: Constructing Self-loops to Adapt Nighttime Semantic Segmentation
• Authors: Fengyi Shen, Zador Pataki, Akhil Gurram, Ziyuan Liu, He Wang, Alois Knoll
• Abstract summary: We propose LoopDA for domain adaptive nighttime semantic segmentation. Our model outperforms prior methods on Dark Zurich and Nighttime Driving datasets for semantic segmentation.
• Score: 5.961294477200831
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Due to the lack of training labels and the difficulty of annotating, dealing with adverse driving conditions such as nighttime has posed a huge challenge to the perception system of autonomous vehicles. Therefore, adapting knowledge from a labelled daytime domain to an unlabelled nighttime domain has been widely researched. In addition to labelled daytime datasets, existing nighttime datasets usually provide nighttime images with corresponding daytime reference images captured at nearby locations for reference. The key challenge is to minimize the performance gap between the two domains. In this paper, we propose LoopDA for domain adaptive nighttime semantic segmentation. It consists of self-loops that result in reconstructing the input data using predicted semantic maps, by rendering them into the encoded features. In a warm-up training stage, the self-loops comprise of an inner-loop and an outer-loop, which are responsible for intra-domain refinement and inter-domain alignment, respectively. To reduce the impact of day-night pose shifts, in the later self-training stage, we propose a co-teaching pipeline that involves an offline pseudo-supervision signal and an online reference-guided signal `DNA' (Day-Night Agreement), bringing substantial benefits to enhance nighttime segmentation. Our model outperforms prior methods on Dark Zurich and Nighttime Driving datasets for semantic segmentation. Code and pretrained models are available at https://github.com/fy-vision/LoopDA.

Related papers

• Exploring Reliable Matching with Phase Enhancement for Night-time Semantic Segmentation [58.180226179087086]
  We propose a novel end-to-end optimized approach, named NightFormer, tailored for night-time semantic segmentation. Specifically, we design a pixel-level texture enhancement module to acquire texture-aware features hierarchically with phase enhancement and amplified attention. Our proposed method performs favorably against state-of-the-art night-time semantic segmentation methods.
  arXiv  Detail & Related papers  (2024-08-25T13:59:31Z)
• PIG: Prompt Images Guidance for Night-Time Scene Parsing [48.35991796324741]
  Unsupervised domain adaptation (UDA) has become the predominant method for studying night scenes. We propose a Night-Focused Network (NFNet) to learn night-specific features from both target domain images and prompt images. We conduct experiments on four night-time datasets: NightCity, NightCity+, Dark Zurich, and ACDC.
  arXiv  Detail & Related papers  (2024-06-15T07:06:19Z)
• Similarity Min-Max: Zero-Shot Day-Night Domain Adaptation [52.923298434948606]
  Low-light conditions not only hamper human visual experience but also degrade the model's performance on downstream vision tasks. This paper challenges a more complicated scenario with border applicability, i.e., zero-shot day-night domain adaptation. We propose a similarity min-max paradigm that considers them under a unified framework.
  arXiv  Detail & Related papers  (2023-07-17T18:50:15Z)
• GPS-GLASS: Learning Nighttime Semantic Segmentation Using Daytime Video and GPS data [15.430918080412518]
  Nighttime semantic segmentation is especially challenging due to a lack of annotated nighttime images. We propose a novel GPS-based training framework for nighttime semantic segmentation. Experimental results demonstrate the effectiveness of the proposed method on several nighttime semantic segmentation datasets.
  arXiv  Detail & Related papers  (2022-07-27T05:05:04Z)
• Cross-Domain Correlation Distillation for Unsupervised Domain Adaptation in Nighttime Semantic Segmentation [17.874336775904272]
  We propose a novel domain adaptation framework via cross-domain correlation distillation, called CCDistill. We extract the content and style knowledge contained in features, calculate the degree of inherent or illumination difference between two images. Experiments on Dark Zurich and ACDC demonstrate that CCDistill achieves the state-of-the-art performance for nighttime semantic segmentation.
  arXiv  Detail & Related papers  (2022-05-02T12:42:04Z)
• Unsupervised Domain Adaptation for Nighttime Aerial Tracking [27.595253191781904]
  This work develops a novel unsupervised domain adaptation framework for nighttime aerial tracking. A unique object discovery approach is provided to generate training patches from raw nighttime tracking videos. With a Transformer day/night feature discriminator, the daytime tracking model is adversarially trained to track at night.
  arXiv  Detail & Related papers  (2022-03-20T12:35:09Z)
• Bi-Mix: Bidirectional Mixing for Domain Adaptive Nighttime Semantic Segmentation [83.97914777313136]
  In autonomous driving, learning a segmentation model that can adapt to various environmental conditions is crucial. In this paper, we study the problem of Domain Adaptive Nighttime Semantic (DANSS), which aims to learn a discriminative nighttime model. We propose a novel Bi-Mix framework for DANSS, which can contribute to both image translation and segmentation adaptation processes.
  arXiv  Detail & Related papers  (2021-11-19T17:39:47Z)
• DANNet: A One-Stage Domain Adaptation Network for Unsupervised Nighttime Semantic Segmentation [18.43890050736093]
  We propose a novel domain adaptation network (DANNet) for nighttime semantic segmentation. It employs an adversarial training with a labeled daytime dataset and an unlabeled dataset that contains coarsely aligned day-night image pairs. Our method achieves state-of-the-art performance for nighttime semantic segmentation.
  arXiv  Detail & Related papers  (2021-04-22T02:49:28Z)
• Map-Guided Curriculum Domain Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation [107.33492779588641]
  We develop a curriculum framework to adapt semantic segmentation models from day to night without using nighttime annotations. We also design a new evaluation framework to address the substantial uncertainty of semantics in nighttime images.
  arXiv  Detail & Related papers  (2020-05-28T16:54:38Z)
• Night-time Scene Parsing with a Large Real Dataset [67.11211537439152]
  We aim to address the night-time scene parsing (NTSP) problem, which has two main challenges. To tackle the scarcity of night-time data, we collect a novel labeled dataset, named it NightCity, of 4,297 real night-time images. We also propose an exposure-aware framework to address the NTSP problem through augmenting the segmentation process with explicitly learned exposure features.
  arXiv  Detail & Related papers  (2020-03-15T18:11:34Z)
• HeatNet: Bridging the Day-Night Domain Gap in Semantic Segmentation with Thermal Images [26.749261270690425]
  Real-world driving scenarios entail adverse environmental conditions such as nighttime illumination or glare. We propose a multimodal semantic segmentation model that can be applied during daytime and nighttime. Besides RGB images, we leverage thermal images, making our network significantly more robust.
  arXiv  Detail & Related papers  (2020-03-10T11:36:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.