DiGA: Distil to Generalize and then Adapt for Domain Adaptive Semantic Segmentation

• URL: http://arxiv.org/abs/2304.02222v1
• Date: Wed, 5 Apr 2023 04:32:02 GMT
• Title: DiGA: Distil to Generalize and then Adapt for Domain Adaptive Semantic Segmentation
• Authors: Fengyi Shen, Akhil Gurram, Ziyuan Liu, He Wang, Alois Knoll
• Abstract summary: Domain adaptive semantic segmentation methods commonly utilize stage-wise training, consisting of a warm-up and a self-training stage. We propose to replace the adversarial training in the warm-up stage by a novel symmetric knowledge distillation module. For the self-training stage, we propose a threshold-free dynamic pseudo-label selection mechanism to ease the aforementioned threshold problem.
• Score: 6.395550661144153
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Domain adaptive semantic segmentation methods commonly utilize stage-wise training, consisting of a warm-up and a self-training stage. However, this popular approach still faces several challenges in each stage: for warm-up, the widely adopted adversarial training often results in limited performance gain, due to blind feature alignment; for self-training, finding proper categorical thresholds is very tricky. To alleviate these issues, we first propose to replace the adversarial training in the warm-up stage by a novel symmetric knowledge distillation module that only accesses the source domain data and makes the model domain generalizable. Surprisingly, this domain generalizable warm-up model brings substantial performance improvement, which can be further amplified via our proposed cross-domain mixture data augmentation technique. Then, for the self-training stage, we propose a threshold-free dynamic pseudo-label selection mechanism to ease the aforementioned threshold problem and make the model better adapted to the target domain. Extensive experiments demonstrate that our framework achieves remarkable and consistent improvements compared to the prior arts on popular benchmarks. Codes and models are available at https://github.com/fy-vision/DiGA

Related papers

• StyDeSty: Min-Max Stylization and Destylization for Single Domain Generalization [85.18995948334592]
  Single domain generalization (single DG) aims at learning a robust model generalizable to unseen domains from only one training domain. State-of-the-art approaches have mostly relied on data augmentations, such as adversarial perturbation and style enhancement, to synthesize new data. We propose emphStyDeSty, which explicitly accounts for the alignment of the source and pseudo domains in the process of data augmentation.
  arXiv  Detail & Related papers  (2024-06-01T02:41:34Z)
• NormAUG: Normalization-guided Augmentation for Domain Generalization [60.159546669021346]
  We propose a simple yet effective method called NormAUG (Normalization-guided Augmentation) for deep learning. Our method introduces diverse information at the feature level and improves the generalization of the main path. In the test stage, we leverage an ensemble strategy to combine the predictions from the auxiliary path of our model, further boosting performance.
  arXiv  Detail & Related papers  (2023-07-25T13:35:45Z)
• Prompting Diffusion Representations for Cross-Domain Semantic Segmentation [101.04326113360342]
  diffusion-pretraining achieves extraordinary domain generalization results for semantic segmentation. We introduce a scene prompt and a prompt randomization strategy to help further disentangle the domain-invariant information when training the segmentation head.
  arXiv  Detail & Related papers  (2023-07-05T09:28:25Z)
• ViDA: Homeostatic Visual Domain Adapter for Continual Test Time Adaptation [48.039156140237615]
  A Continual Test-Time Adaptation task is proposed to adapt the pre-trained model to continually changing target domains. We design a Visual Domain Adapter (ViDA) for CTTA, explicitly handling both domain-specific and domain-shared knowledge. Our proposed method achieves state-of-the-art performance in both classification and segmentation CTTA tasks.
  arXiv  Detail & Related papers  (2023-06-07T11:18:53Z)
• Continual Unsupervised Domain Adaptation for Semantic Segmentation using a Class-Specific Transfer [9.46677024179954]
  segmentation models do not generalize to unseen domains. We propose a light-weight style transfer framework that incorporates two class-conditional AdaIN layers. We extensively validate our approach on a synthetic sequence and further propose a challenging sequence consisting of real domains.
  arXiv  Detail & Related papers  (2022-08-12T21:30:49Z)
• Learning Instance-Specific Adaptation for Cross-Domain Segmentation [79.61787982393238]
  We propose a test-time adaptation method for cross-domain image segmentation. Given a new unseen instance at test time, we adapt a pre-trained model by conducting instance-specific BatchNorm calibration.
  arXiv  Detail & Related papers  (2022-03-30T17:59:45Z)
• Style Mixing and Patchwise Prototypical Matching for One-Shot Unsupervised Domain Adaptive Semantic Segmentation [21.01132797297286]
  In one-shot unsupervised domain adaptation, segmentors only see one unlabeled target image during training. We propose a new OSUDA method that can effectively relieve such computational burden. Our method achieves new state-of-the-art performance on two commonly used benchmarks for domain adaptive semantic segmentation.
  arXiv  Detail & Related papers  (2021-12-09T02:47:46Z)
• Shallow Features Guide Unsupervised Domain Adaptation for Semantic Segmentation at Class Boundaries [21.6953660626021]
  Deep neural networks fail to generalize towards new domains when performing synthetic-to-real adaptation. In this work, we present a novel low-level adaptation strategy that allows us to obtain sharp predictions. We also introduce an effective data augmentation that alleviates the noise typically present at semantic boundaries when employing pseudo-labels for self-training.
  arXiv  Detail & Related papers  (2021-10-06T15:05:48Z)
• Stagewise Unsupervised Domain Adaptation with Adversarial Self-Training for Road Segmentation of Remote Sensing Images [93.50240389540252]
  Road segmentation from remote sensing images is a challenging task with wide ranges of application potentials. We propose a novel stagewise domain adaptation model called RoadDA to address the domain shift (DS) issue in this field. Experiment results on two benchmarks demonstrate that RoadDA can efficiently reduce the domain gap and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-08-28T09:29:14Z)
• Unsupervised Intra-domain Adaptation for Semantic Segmentation through Self-Supervision [73.76277367528657]
  Convolutional neural network-based approaches have achieved remarkable progress in semantic segmentation. To cope with this limitation, automatically annotated data generated from graphic engines are used to train segmentation models. We propose a two-step self-supervised domain adaptation approach to minimize the inter-domain and intra-domain gap together.
  arXiv  Detail & Related papers  (2020-04-16T15:24:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.