Hard-aware Instance Adaptive Self-training for Unsupervised Cross-domain Semantic Segmentation

• URL: http://arxiv.org/abs/2302.06992v2
• Date: Tue, 25 Mar 2025 03:37:02 GMT
• Title: Hard-aware Instance Adaptive Self-training for Unsupervised Cross-domain Semantic Segmentation
• Authors: Chuang Zhu, Kebin Liu, Wenqi Tang, Ke Mei, Jiaqi Zou, Tiejun Huang,
• Abstract summary: We propose a hard-aware instance adaptive self-training framework for UDA on the task of semantic segmentation.<n>To effectively improve the quality and diversity of pseudo-labels, we develop a novel pseudo-label generation strategy.<n>Our method is so concise and efficient that it is easy to be generalized to other UDA methods.
• Score: 23.65286902893462
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The divergence between labeled training data and unlabeled testing data is a significant challenge for recent deep learning models. Unsupervised domain adaptation (UDA) attempts to solve such problem. Recent works show that self-training is a powerful approach to UDA. However, existing methods have difficulty in balancing the scalability and performance. In this paper, we propose a hard-aware instance adaptive self-training framework for UDA on the task of semantic segmentation. To effectively improve the quality and diversity of pseudo-labels, we develop a novel pseudo-label generation strategy with an instance adaptive selector. We further enrich the hard class pseudo-labels with inter-image information through a skillfully designed hard-aware pseudo-label augmentation. Besides, we propose the region-adaptive regularization to smooth the pseudo-label region and sharpen the non-pseudo-label region. For the non-pseudo-label region, consistency constraint is also constructed to introduce stronger supervision signals during model optimization. Our method is so concise and efficient that it is easy to be generalized to other UDA methods. Experiments on GTA5 to Cityscapes, SYNTHIA to Cityscapes, and Cityscapes to Oxford RobotCar demonstrate the superior performance of our approach compared with the state-of-the-art methods. Our codes are available at https://github.com/bupt-ai-cz/HIAST.

Related papers

• Pseudolabel guided pixels contrast for domain adaptive semantic segmentation [0.9831489366502301]
  Unsupervised domain adaptation (UDA) for semantic segmentation is a technique that uses virtual data with labels to train a model and adapts it to real data without labels. Some recent works use contrastive learning, which is a powerful method for self-supervised learning, to help with this technique. We propose a novel framework called Pseudo-label Guided Pixel Contrast (PGPC), which overcomes the disadvantages of previous methods.
  arXiv  Detail & Related papers  (2025-01-15T03:25:25Z)
• The Last Mile to Supervised Performance: Semi-Supervised Domain Adaptation for Semantic Segmentation [51.77968964691317]
  We study the promising setting of Semi-Supervised Domain Adaptation (SSDA) We propose a simple SSDA framework that combines consistency regularization, pixel contrastive learning, and self-training to effectively utilize a few target-domain labels. Our method outperforms prior art in the popular GTA-to-Cityscapes benchmark and shows that as little as 50 target labels can suffice to achieve near-supervised performance.
  arXiv  Detail & Related papers  (2024-11-27T20:07:42Z)
• Towards Modality-agnostic Label-efficient Segmentation with Entropy-Regularized Distribution Alignment [62.73503467108322]
  This topic is widely studied in 3D point cloud segmentation due to the difficulty of annotating point clouds densely. Until recently, pseudo-labels have been widely employed to facilitate training with limited ground-truth labels. Existing pseudo-labeling approaches could suffer heavily from the noises and variations in unlabelled data. We propose a novel learning strategy to regularize the pseudo-labels generated for training, thus effectively narrowing the gaps between pseudo-labels and model predictions.
  arXiv  Detail & Related papers  (2024-08-29T13:31:15Z)
• ECAP: Extensive Cut-and-Paste Augmentation for Unsupervised Domain Adaptive Semantic Segmentation [4.082799056366928]
  We propose an extensive cut-and-paste strategy (ECAP) to leverage reliable pseudo-labels through data augmentation. ECAP maintains a memory bank of pseudo-labeled target samples throughout training and cut-and-pastes the most confident ones onto the current training batch. We implement ECAP on top of the recent method MIC and boost its performance on two synthetic-to-real domain adaptation benchmarks.
  arXiv  Detail & Related papers  (2024-03-06T17:06:07Z)
• SAM4UDASS: When SAM Meets Unsupervised Domain Adaptive Semantic Segmentation in Intelligent Vehicles [27.405213492173186]
  We introduce SAM4UDASS, a novel approach that incorporates the Segment Anything Model (SAM) into self-training UDA methods for refining pseudo-labels. It involves Semantic-Guided Mask Labeling, which assigns semantic labels to unlabeled SAM masks using UDA pseudo-labels. It brings more than 3% mIoU gains on GTA5-to-Cityscapes, SYNTHIA-to-Cityscapes, and Cityscapes-to-ACDC when using DAFormer and achieves SOTA when using MIC.
  arXiv  Detail & Related papers  (2023-11-22T08:29:45Z)
• Unsupervised Domain Adaptation for Semantic Segmentation with Pseudo Label Self-Refinement [9.69089112870202]
  We propose an auxiliary pseudo-label refinement network (PRN) for online refining of the pseudo labels and also localizing the pixels whose predicted labels are likely to be noisy. We evaluate our approach on benchmark datasets with three different domain shifts, and our approach consistently performs significantly better than the previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-10-25T20:31:07Z)
• All Points Matter: Entropy-Regularized Distribution Alignment for Weakly-supervised 3D Segmentation [67.30502812804271]
  Pseudo-labels are widely employed in weakly supervised 3D segmentation tasks where only sparse ground-truth labels are available for learning. We propose a novel learning strategy to regularize the generated pseudo-labels and effectively narrow the gaps between pseudo-labels and model predictions.
  arXiv  Detail & Related papers  (2023-05-25T08:19:31Z)
• Unsupervised Domain Adaptive Salient Object Detection Through Uncertainty-Aware Pseudo-Label Learning [104.00026716576546]
  We propose to learn saliency from synthetic but clean labels, which naturally has higher pixel-labeling quality without the effort of manual annotations. We show that our proposed method outperforms the existing state-of-the-art deep unsupervised SOD methods on several benchmark datasets.
  arXiv  Detail & Related papers  (2022-02-26T16:03:55Z)
• STRUDEL: Self-Training with Uncertainty Dependent Label Refinement across Domains [4.812718493682454]
  We propose an unsupervised domain adaptation (UDA) approach for white matter hyperintensity (WMH) segmentation. We propose to predict the uncertainty of pseudo labels and integrate it in the training process with an uncertainty-guided loss function to highlight labels with high certainty. Our results on WMH segmentation across datasets demonstrate the significant improvement of STRUDEL with respect to standard self-training.
  arXiv  Detail & Related papers  (2021-04-23T13:46:26Z)
• Semi-Supervised Domain Adaptation with Prototypical Alignment and Consistency Learning [86.6929930921905]
  This paper studies how much it can help address domain shifts if we further have a few target samples labeled. To explore the full potential of landmarks, we incorporate a prototypical alignment (PA) module which calculates a target prototype for each class from the landmarks. Specifically, we severely perturb the labeled images, making PA non-trivial to achieve and thus promoting model generalizability.
  arXiv  Detail & Related papers  (2021-04-19T08:46:08Z)
• Cycle Self-Training for Domain Adaptation [85.14659717421533]
  Cycle Self-Training (CST) is a principled self-training algorithm that enforces pseudo-labels to generalize across domains. CST recovers target ground truth, while both invariant feature learning and vanilla self-training fail. Empirical results indicate that CST significantly improves over prior state-of-the-arts in standard UDA benchmarks.
  arXiv  Detail & Related papers  (2021-03-05T10:04:25Z)
• Selective Pseudo-Labeling with Reinforcement Learning for Semi-Supervised Domain Adaptation [116.48885692054724]
  We propose a reinforcement learning based selective pseudo-labeling method for semi-supervised domain adaptation. We develop a deep Q-learning model to select both accurate and representative pseudo-labeled instances. Our proposed method is evaluated on several benchmark datasets for SSDA, and demonstrates superior performance to all the comparison methods.
  arXiv  Detail & Related papers  (2020-12-07T03:37:38Z)
• PseudoSeg: Designing Pseudo Labels for Semantic Segmentation [78.35515004654553]
  We present a re-design of pseudo-labeling to generate structured pseudo labels for training with unlabeled or weakly-labeled data. We demonstrate the effectiveness of the proposed pseudo-labeling strategy in both low-data and high-data regimes.
  arXiv  Detail & Related papers  (2020-10-19T17:59:30Z)
• Instance Adaptive Self-Training for Unsupervised Domain Adaptation [19.44504738538047]
  We propose an instance adaptive self-training framework for UDA on the task of semantic segmentation. To effectively improve the quality of pseudo-labels, we develop a novel pseudo-label generation strategy with an instance adaptive selector. Our method is so concise and efficient that it is easy to be generalized to other unsupervised domain adaptation methods.
  arXiv  Detail & Related papers  (2020-08-27T15:50:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.