DTBS: Dual-Teacher Bi-directional Self-training for Domain Adaptation in Nighttime Semantic Segmentation

• URL: http://arxiv.org/abs/2401.01066v1
• Date: Tue, 2 Jan 2024 06:56:57 GMT
• Title: DTBS: Dual-Teacher Bi-directional Self-training for Domain Adaptation in Nighttime Semantic Segmentation
• Authors: Fanding Huang, Zihao Yao and Wenhui Zhou
• Abstract summary: Nighttime conditions pose a significant challenge for autonomous vehicle perception systems. Unsupervised domain adaptation (UDA) has been widely applied to semantic segmentation on such images. We introduce a one-stage Dual-Teacher Bi-directional Self-training (DTBS) framework for smooth knowledge transfer and feedback.
• Score: 1.7205106391379026
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Due to the poor illumination and the difficulty in annotating, nighttime conditions pose a significant challenge for autonomous vehicle perception systems. Unsupervised domain adaptation (UDA) has been widely applied to semantic segmentation on such images to adapt models from normal conditions to target nighttime-condition domains. Self-training (ST) is a paradigm in UDA, where a momentum teacher is utilized for pseudo-label prediction, but a confirmation bias issue exists. Because the one-directional knowledge transfer from a single teacher is insufficient to adapt to a large domain shift. To mitigate this issue, we propose to alleviate domain gap by incrementally considering style influence and illumination change. Therefore, we introduce a one-stage Dual-Teacher Bi-directional Self-training (DTBS) framework for smooth knowledge transfer and feedback. Based on two teacher models, we present a novel pipeline to respectively decouple style and illumination shift. In addition, we propose a new Re-weight exponential moving average (EMA) to merge the knowledge of style and illumination factors, and provide feedback to the student model. In this way, our method can be embedded in other UDA methods to enhance their performance. For example, the Cityscapes to ACDC night task yielded 53.8 mIoU (\%), which corresponds to an improvement of +5\% over the previous state-of-the-art. The code is available at \url{https://github.com/hf618/DTBS}.

Related papers

• Enhancing Domain Adaptation through Prompt Gradient Alignment [16.618313165111793]
  We develop a line of works based on prompt learning to learn both domain-invariant and specific features. We cast UDA as a multiple-objective optimization problem in which each objective is represented by a domain loss. Our method consistently surpasses other prompt-based baselines by a large margin on different UDA benchmarks.
  arXiv  Detail & Related papers  (2024-06-13T17:40:15Z)
• Prompt-based Distribution Alignment for Unsupervised Domain Adaptation [42.77798810726824]
  We experimentally demonstrate that the unsupervised-trained visual-language models (VLMs) can significantly reduce the distribution discrepancy between source and target domains. A major challenge for directly deploying such models on downstream UDA tasks is prompt engineering. We propose a Prompt-based Distribution Alignment (PDA) method to incorporate the domain knowledge into prompt learning.
  arXiv  Detail & Related papers  (2023-12-15T06:15:04Z)
• T-UDA: Temporal Unsupervised Domain Adaptation in Sequential Point Clouds [2.5291108878852864]
  unsupervised domain adaptation (UDA) methods adapt models trained on one (source) domain with annotations available to another (target) domain for which only unannotated data are available. We introduce a novel domain adaptation method that leverages the best of both trends. Dubbed T-UDA for "temporal UDA", such a combination yields massive performance gains for the task of 3D semantic segmentation of driving scenes.
  arXiv  Detail & Related papers  (2023-09-15T10:47:12Z)
• A denoised Mean Teacher for domain adaptive point cloud registration [81.43344461130474]
  Point cloud-based medical registration promises increased computational efficiency, robustness to intensity shifts, and anonymity preservation. Supervised training on synthetic deformations is an alternative but, in turn, suffers from the domain gap to the real domain. We present a denoised teacher-student paradigm for point cloud registration, comprising two complementary denoising strategies.
  arXiv  Detail & Related papers  (2023-06-26T15:03:47Z)
• Focus on Your Target: A Dual Teacher-Student Framework for Domain-adaptive Semantic Segmentation [210.46684938698485]
  We study unsupervised domain adaptation (UDA) for semantic segmentation. We find that, by decreasing/increasing the proportion of training samples from the target domain, the 'learning ability' is strengthened/weakened. We propose a novel dual teacher-student (DTS) framework and equip it with a bidirectional learning strategy.
  arXiv  Detail & Related papers  (2023-03-16T05:04:10Z)
• Prior Knowledge Guided Unsupervised Domain Adaptation [82.9977759320565]
  We propose a Knowledge-guided Unsupervised Domain Adaptation (KUDA) setting where prior knowledge about the target class distribution is available. In particular, we consider two specific types of prior knowledge about the class distribution in the target domain: Unary Bound and Binary Relationship. We propose a rectification module that uses such prior knowledge to refine model generated pseudo labels.
  arXiv  Detail & Related papers  (2022-07-18T18:41:36Z)
• Target and Task specific Source-Free Domain Adaptive Image Segmentation [73.78898054277538]
  We propose a two-stage approach for source-free domain adaptive image segmentation. We focus on generating target-specific pseudo labels while suppressing high entropy regions. In the second stage, we focus on adapting the network for task-specific representation.
  arXiv  Detail & Related papers  (2022-03-29T17:50:22Z)
• DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation [99.88539409432916]
  We study the unsupervised domain adaptation (UDA) process. We propose a novel UDA method, DAFormer, based on the benchmark results. DAFormer significantly improves the state-of-the-art performance by 10.8 mIoU for GTA->Cityscapes and 5.4 mIoU for Synthia->Cityscapes.
  arXiv  Detail & Related papers  (2021-11-29T19:00:46Z)
• Stagewise Unsupervised Domain Adaptation with Adversarial Self-Training for Road Segmentation of Remote Sensing Images [93.50240389540252]
  Road segmentation from remote sensing images is a challenging task with wide ranges of application potentials. We propose a novel stagewise domain adaptation model called RoadDA to address the domain shift (DS) issue in this field. Experiment results on two benchmarks demonstrate that RoadDA can efficiently reduce the domain gap and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-08-28T09:29:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.