Adversarial Semantic Hallucination for Domain Generalized Semantic Segmentation

• URL: http://arxiv.org/abs/2106.04144v1
• Date: Tue, 8 Jun 2021 07:07:45 GMT
• Title: Adversarial Semantic Hallucination for Domain Generalized Semantic Segmentation
• Authors: Gabriel Tjio, Ping Liu, Joey Tianyi Zhou, Rick Siow Mong Goh
• Abstract summary: We propose an adversarial hallucination approach, which combines a class-wise hallucination module and a semantic segmentation module. Experiments on state of the art domain adaptation work demonstrate the efficacy of our proposed method when no target domain data are available for training.
• Score: 50.14933487082085
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Convolutional neural networks may perform poorly when the test and train data are from different domains. While this problem can be mitigated by using the target domain data to align the source and target domain feature representations, the target domain data may be unavailable due to privacy concerns. Consequently, there is a need for methods that generalize well without access to target domain data during training. In this work, we propose an adversarial hallucination approach, which combines a class-wise hallucination module and a semantic segmentation module. Since the segmentation performance varies across different classes, we design a semantic-conditioned style hallucination layer to adaptively stylize each class. The classwise stylization parameters are generated from the semantic knowledge in the segmentation probability maps of the source domain image. Both modules compete adversarially, with the hallucination module generating increasingly 'difficult' style images to challenge the segmentation module. In response, the segmentation module improves its performance as it is trained with generated samples at an appropriate class-wise difficulty level. Experiments on state of the art domain adaptation work demonstrate the efficacy of our proposed method when no target domain data are available for training.

Related papers

• AdaptDiff: Cross-Modality Domain Adaptation via Weak Conditional Semantic Diffusion for Retinal Vessel Segmentation [10.958821619282748]
  We present an unsupervised domain adaptation (UDA) method named AdaptDiff. It enables a retinal vessel segmentation network trained on fundus photography (FP) to produce satisfactory results on unseen modalities. Our results demonstrate a significant improvement in segmentation performance across all unseen datasets.
  arXiv  Detail & Related papers  (2024-10-06T23:04:29Z)
• Phrase Grounding-based Style Transfer for Single-Domain Generalized Object Detection [109.58348694132091]
  Single-domain generalized object detection aims to enhance a model's generalizability to multiple unseen target domains. This is a practical yet challenging task as it requires the model to address domain shift without incorporating target domain data into training. We propose a novel phrase grounding-based style transfer approach for the task.
  arXiv  Detail & Related papers  (2024-02-02T10:48:43Z)
• Domain-invariant Prototypes for Semantic Segmentation [30.932130453313537]
  We present an easy-to-train framework that learns domain-invariant prototypes for domain adaptive semantic segmentation. Our method involves only one-stage training and does not need to be trained on large-scale un-annotated target images.
  arXiv  Detail & Related papers  (2022-08-12T02:21:05Z)
• Unsupervised Domain Adaptation for Point Cloud Semantic Segmentation via Graph Matching [14.876681993079062]
  We propose a graph-based framework to explore the local-level feature alignment between the two domains. We also formulate a category-guided contrastive loss to guide the segmentation model to learn discriminative features on the target domain.
  arXiv  Detail & Related papers  (2022-08-09T02:30:15Z)
• SPCL: A New Framework for Domain Adaptive Semantic Segmentation via Semantic Prototype-based Contrastive Learning [6.705297811617307]
  Domain adaptation can help in transferring knowledge from a labeled source domain to an unlabeled target domain. We propose a novel semantic prototype-based contrastive learning framework for fine-grained class alignment. Our method is easy to implement and attains superior results compared to state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-11-24T09:26:07Z)
• Domain Adaptive Semantic Segmentation without Source Data [50.18389578589789]
  We investigate domain adaptive semantic segmentation without source data, which assumes that the model is pre-trained on the source domain. We propose an effective framework for this challenging problem with two components: positive learning and negative learning. Our framework can be easily implemented and incorporated with other methods to further enhance the performance.
  arXiv  Detail & Related papers  (2021-10-13T04:12:27Z)
• Semi-supervised Meta-learning with Disentanglement for Domain-generalised Medical Image Segmentation [15.351113774542839]
  Generalising models to new data from new centres (termed here domains) remains a challenge. We propose a novel semi-supervised meta-learning framework with disentanglement. We show that the proposed method is robust on different segmentation tasks and achieves state-of-the-art generalisation performance on two public benchmarks.
  arXiv  Detail & Related papers  (2021-06-24T19:50:07Z)
• Contrastive Learning and Self-Training for Unsupervised Domain Adaptation in Semantic Segmentation [71.77083272602525]
  UDA attempts to provide efficient knowledge transfer from a labeled source domain to an unlabeled target domain. We propose a contrastive learning approach that adapts category-wise centroids across domains. We extend our method with self-training, where we use a memory-efficient temporal ensemble to generate consistent and reliable pseudo-labels.
  arXiv  Detail & Related papers  (2021-05-05T11:55:53Z)
• Phase Consistent Ecological Domain Adaptation [76.75730500201536]
  We focus on the task of semantic segmentation, where annotated synthetic data are aplenty, but annotating real data is laborious. The first criterion, inspired by visual psychophysics, is that the map between the two image domains be phase-preserving. The second criterion aims to leverage ecological statistics, or regularities in the scene which are manifest in any image of it, regardless of the characteristics of the illuminant or the imaging sensor.
  arXiv  Detail & Related papers  (2020-04-10T06:58:03Z)
• Alleviating Semantic-level Shift: A Semi-supervised Domain Adaptation Method for Semantic Segmentation [97.8552697905657]
  A key challenge of this task is how to alleviate the data distribution discrepancy between the source and target domains. We propose Alleviating Semantic-level Shift (ASS), which can successfully promote the distribution consistency from both global and local views. We apply our ASS to two domain adaptation tasks, from GTA5 to Cityscapes and from Synthia to Cityscapes.
  arXiv  Detail & Related papers  (2020-04-02T03:25:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.