DynAlign: Unsupervised Dynamic Taxonomy Alignment for Cross-Domain Segmentation

• URL: http://arxiv.org/abs/2501.16410v1
• Date: Mon, 27 Jan 2025 18:57:19 GMT
• Title: DynAlign: Unsupervised Dynamic Taxonomy Alignment for Cross-Domain Segmentation
• Authors: Han Sun, Rui Gong, Ismail Nejjar, Olga Fink,
• Abstract summary: We introduce DynAlign, a framework that integrates UDA with foundation models to bridge the image-level and label-level domain gaps. Our approach leverages prior semantic knowledge to align source categories with target categories that can be novel, more fine-grained, or named differently. DynAlign generates accurate predictions in a new target label space without requiring any manual annotations.
• Score: 15.303659468173334
• License:
• Abstract: Current unsupervised domain adaptation (UDA) methods for semantic segmentation typically assume identical class labels between the source and target domains. This assumption ignores the label-level domain gap, which is common in real-world scenarios, thus limiting their ability to identify finer-grained or novel categories without requiring extensive manual annotation. A promising direction to address this limitation lies in recent advancements in foundation models, which exhibit strong generalization abilities due to their rich prior knowledge. However, these models often struggle with domain-specific nuances and underrepresented fine-grained categories. To address these challenges, we introduce DynAlign, a framework that integrates UDA with foundation models to bridge both the image-level and label-level domain gaps. Our approach leverages prior semantic knowledge to align source categories with target categories that can be novel, more fine-grained, or named differently (e.g., vehicle to {car, truck, bus}). Foundation models are then employed for precise segmentation and category reassignment. To further enhance accuracy, we propose a knowledge fusion approach that dynamically adapts to varying scene contexts. DynAlign generates accurate predictions in a new target label space without requiring any manual annotations, allowing seamless adaptation to new taxonomies through either model retraining or direct inference. Experiments on the street scene semantic segmentation benchmarks GTA to Mapillary Vistas and GTA to IDD validate the effectiveness of our approach, achieving a significant improvement over existing methods. Our code will be publicly available.

Related papers

• Cross-Domain Semantic Segmentation on Inconsistent Taxonomy using VLMs [1.4182672294839365]
  Cross-Domain Semantic on Inconsistent taxonomy using Vision Language Models (CSI) This paper introduces a novel approach, Cross-Domain Semantic on Inconsistent taxonomy using Vision Language Models (CSI) It effectively performs domain-adaptive semantic segmentation even in situations of source-target class mismatches.
  arXiv  Detail & Related papers  (2024-08-05T06:32:20Z)
• Pulling Target to Source: A New Perspective on Domain Adaptive Semantic Segmentation [80.1412989006262]
  Domain adaptive semantic segmentation aims to transfer knowledge from a labeled source domain to an unlabeled target domain. We propose T2S-DA, which we interpret as a form of pulling Target to Source for Domain Adaptation.
  arXiv  Detail & Related papers  (2023-05-23T07:09:09Z)
• Continual Unsupervised Domain Adaptation for Semantic Segmentation using a Class-Specific Transfer [9.46677024179954]
  segmentation models do not generalize to unseen domains. We propose a light-weight style transfer framework that incorporates two class-conditional AdaIN layers. We extensively validate our approach on a synthetic sequence and further propose a challenging sequence consisting of real domains.
  arXiv  Detail & Related papers  (2022-08-12T21:30:49Z)
• Prototypical Contrast Adaptation for Domain Adaptive Semantic Segmentation [52.63046674453461]
  Prototypical Contrast Adaptation (ProCA) is a contrastive learning method for unsupervised domain adaptive semantic segmentation. ProCA incorporates inter-class information into class-wise prototypes, and adopts the class-centered distribution alignment for adaptation.
  arXiv  Detail & Related papers  (2022-07-14T04:54:26Z)
• TADA: Taxonomy Adaptive Domain Adaptation [143.68890984935726]
  Traditional domain adaptation addresses the task of adapting a model to a novel target domain under limited supervision. We introduce the more general taxonomy adaptive domain adaptation problem, allowing for inconsistent between the two domains. On the label-level, we employ a bilateral mixed sampling strategy to augment the target domain, and a relabelling method to unify and align the label spaces.
  arXiv  Detail & Related papers  (2021-09-10T11:58:56Z)
• On Universal Black-Box Domain Adaptation [53.7611757926922]
  We study an arguably least restrictive setting of domain adaptation in a sense of practical deployment. Only the interface of source model is available to the target domain, and where the label-space relations between the two domains are allowed to be different and unknown. We propose to unify them into a self-training framework, regularized by consistency of predictions in local neighborhoods of target samples.
  arXiv  Detail & Related papers  (2021-04-10T02:21:09Z)
• Get away from Style: Category-Guided Domain Adaptation for Semantic Segmentation [15.002381934551359]
  Unsupervised domain adaptation (UDA) becomes more and more popular in tackling real-world problems without ground truth of the target domain. In this paper, we focus on UDA for semantic segmentation task. We propose a style-independent content feature extraction mechanism to keep the style information of extracted features in the similar space. Secondly, to keep the balance of pseudo labels on each category, we propose a category-guided threshold mechanism to choose category-wise pseudo labels for self-supervised learning.
  arXiv  Detail & Related papers  (2021-03-29T10:00:50Z)
• Your Classifier can Secretly Suffice Multi-Source Domain Adaptation [72.47706604261992]
  Multi-Source Domain Adaptation (MSDA) deals with the transfer of task knowledge from multiple labeled source domains to an unlabeled target domain. We present a different perspective to MSDA wherein deep models are observed to implicitly align the domains under label supervision.
  arXiv  Detail & Related papers  (2021-03-20T12:44:13Z)
• Domain Adaptive Semantic Segmentation Using Weak Labels [115.16029641181669]
  We propose a novel framework for domain adaptation in semantic segmentation with image-level weak labels in the target domain. We develop a weak-label classification module to enforce the network to attend to certain categories. In experiments, we show considerable improvements with respect to the existing state-of-the-arts in UDA and present a new benchmark in the WDA setting.
  arXiv  Detail & Related papers  (2020-07-30T01:33:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.