Exploring Semantic Consistency and Style Diversity for Domain Generalized Semantic Segmentation

• URL: http://arxiv.org/abs/2412.12050v1
• Date: Mon, 16 Dec 2024 18:20:06 GMT
• Title: Exploring Semantic Consistency and Style Diversity for Domain Generalized Semantic Segmentation
• Authors: Hongwei Niu, Linhuang Xie, Jianghang Lin, Shengchuan Zhang,
• Abstract summary: Domain Generalized Semantic aims to enhance the generalization of semantic segmentation across unknown target domains. We introduce SCSD for Semantic Consistency prediction and Style Diversity generalization. SCSD significantly outperforms existing state-of-theart methods.
• Score: 4.850207292777464
• License:
• Abstract: Domain Generalized Semantic Segmentation (DGSS) seeks to utilize source domain data exclusively to enhance the generalization of semantic segmentation across unknown target domains. Prevailing studies predominantly concentrate on feature normalization and domain randomization, these approaches exhibit significant limitations. Feature normalization-based methods tend to confuse semantic features in the process of constraining the feature space distribution, resulting in classification misjudgment. Domain randomization-based methods frequently incorporate domain-irrelevant noise due to the uncontrollability of style transformations, resulting in segmentation ambiguity. To address these challenges, we introduce a novel framework, named SCSD for Semantic Consistency prediction and Style Diversity generalization. It comprises three pivotal components: Firstly, a Semantic Query Booster is designed to enhance the semantic awareness and discrimination capabilities of object queries in the mask decoder, enabling cross-domain semantic consistency prediction. Secondly, we develop a Text-Driven Style Transform module that utilizes domain difference text embeddings to controllably guide the style transformation of image features, thereby increasing inter-domain style diversity. Lastly, to prevent the collapse of similar domain feature spaces, we introduce a Style Synergy Optimization mechanism that fortifies the separation of inter-domain features and the aggregation of intra-domain features by synergistically weighting style contrastive loss and style aggregation loss. Extensive experiments demonstrate that the proposed SCSD significantly outperforms existing state-of-theart methods. Notably, SCSD trained on GTAV achieved an average of 49.11 mIoU on the four unseen domain datasets, surpassing the previous state-of-the-art method by +4.08 mIoU. Code is available at https://github.com/nhw649/SCSD.

Related papers

• A Universal Degradation-based Bridging Technique for Domain Adaptive Semantic Segmentation [32.1141872384702]
  DiDA is a universal degradation-based bridging technique formalized as a diffusion forward process. As a plug-and-play solution, DiDA supports various degradation operations and seamlessly integrates with existing UDA methods.
  arXiv  Detail & Related papers  (2024-12-13T18:35:27Z)
• Semantic-Rearrangement-Based Multi-Level Alignment for Domain Generalized Segmentation [11.105659621713855]
  We argue that different local semantic regions perform different visual characteristics from the source domain to the target domain. We propose the Semantic-Rearrangement-based Multi-Level Alignment (SRMA) to overcome this problem.
  arXiv  Detail & Related papers  (2024-04-21T16:05:38Z)
• Decomposition-based Unsupervised Domain Adaptation for Remote Sensing Image Semantic Segmentation [30.606689882397223]
  Unsupervised domain adaptation (UDA) techniques are vital for semantic segmentation in geosciences. Most existing UDA methods, which focus on domain alignment at the high-level feature space, struggle to simultaneously retain local spatial details and global contextual semantics. A novel decomposition scheme is proposed to guide domain-invariant representation learning.
  arXiv  Detail & Related papers  (2024-04-06T07:13:49Z)
• Amplitude Spectrum Transformation for Open Compound Domain Adaptive Semantic Segmentation [62.68759523116924]
  Open compound domain adaptation (OCDA) has emerged as a practical adaptation setting. We propose a novel feature space Amplitude Spectrum Transformation (AST)
  arXiv  Detail & Related papers  (2022-02-09T05:40:34Z)
• Structured Latent Embeddings for Recognizing Unseen Classes in Unseen Domains [108.11746235308046]
  We propose a novel approach that learns domain-agnostic structured latent embeddings by projecting images from different domains. Our experiments on the challenging DomainNet and DomainNet-LS benchmarks show the superiority of our approach over existing methods.
  arXiv  Detail & Related papers  (2021-07-12T17:57:46Z)
• More Separable and Easier to Segment: A Cluster Alignment Method for Cross-Domain Semantic Segmentation [41.81843755299211]
  We propose a new UDA semantic segmentation approach based on domain assumption closeness to alleviate the above problems. Specifically, a prototype clustering strategy is applied to cluster pixels with the same semantic, which will better maintain associations among target domain pixels. Experiments conducted on GTA5 and SYNTHIA proved the effectiveness of our method.
  arXiv  Detail & Related papers  (2021-05-07T10:24:18Z)
• Generalizable Representation Learning for Mixture Domain Face Anti-Spoofing [53.82826073959756]
  Face anti-spoofing approach based on domain generalization(DG) has drawn growing attention due to its robustness forunseen scenarios. We propose domain dy-namic adjustment meta-learning (D2AM) without using do-main labels. To overcome the limitation, we propose domain dy-namic adjustment meta-learning (D2AM) without using do-main labels.
  arXiv  Detail & Related papers  (2021-05-06T06:04:59Z)
• Cluster, Split, Fuse, and Update: Meta-Learning for Open Compound Domain Adaptive Semantic Segmentation [102.42638795864178]
  We propose a principled meta-learning based approach to OCDA for semantic segmentation. We cluster target domain into multiple sub-target domains by image styles, extracted in an unsupervised manner. A meta-learner is thereafter deployed to learn to fuse sub-target domain-specific predictions, conditioned upon the style code. We learn to online update the model by model-agnostic meta-learning (MAML) algorithm, thus to further improve generalization.
  arXiv  Detail & Related papers  (2020-12-15T13:21:54Z)
• Cross-Domain Grouping and Alignment for Domain Adaptive Semantic Segmentation [74.3349233035632]
  Existing techniques to adapt semantic segmentation networks across the source and target domains within deep convolutional neural networks (CNNs) do not consider an inter-class variation within the target domain itself or estimated category. We introduce a learnable clustering module, and a novel domain adaptation framework called cross-domain grouping and alignment. Our method consistently boosts the adaptation performance in semantic segmentation, outperforming the state-of-the-arts on various domain adaptation settings.
  arXiv  Detail & Related papers  (2020-12-15T11:36:21Z)
• Affinity Space Adaptation for Semantic Segmentation Across Domains [57.31113934195595]
  In this paper, we address the problem of unsupervised domain adaptation (UDA) in semantic segmentation. Motivated by the fact that source and target domain have invariant semantic structures, we propose to exploit such invariance across domains. We develop two affinity space adaptation strategies: affinity space cleaning and adversarial affinity space alignment.
  arXiv  Detail & Related papers  (2020-09-26T10:28:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.