Cross-Domain Semantic Segmentation on Inconsistent Taxonomy using VLMs

• URL: http://arxiv.org/abs/2408.02261v1
• Date: Mon, 5 Aug 2024 06:32:20 GMT
• Title: Cross-Domain Semantic Segmentation on Inconsistent Taxonomy using VLMs
• Authors: Jeongkee Lim, Yusung Kim,
• Abstract summary: Cross-Domain Semantic on Inconsistent taxonomy using Vision Language Models (CSI) This paper introduces a novel approach, Cross-Domain Semantic on Inconsistent taxonomy using Vision Language Models (CSI) It effectively performs domain-adaptive semantic segmentation even in situations of source-target class mismatches.
• Score: 1.4182672294839365
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The challenge of semantic segmentation in Unsupervised Domain Adaptation (UDA) emerges not only from domain shifts between source and target images but also from discrepancies in class taxonomies across domains. Traditional UDA research assumes consistent taxonomy between the source and target domains, thereby limiting their ability to recognize and adapt to the taxonomy of the target domain. This paper introduces a novel approach, Cross-Domain Semantic Segmentation on Inconsistent Taxonomy using Vision Language Models (CSI), which effectively performs domain-adaptive semantic segmentation even in situations of source-target class mismatches. CSI leverages the semantic generalization potential of Visual Language Models (VLMs) to create synergy with previous UDA methods. It leverages segment reasoning obtained through traditional UDA methods, combined with the rich semantic knowledge embedded in VLMs, to relabel new classes in the target domain. This approach allows for effective adaptation to extended taxonomies without requiring any ground truth label for the target domain. Our method has shown to be effective across various benchmarks in situations of inconsistent taxonomy settings (coarse-to-fine taxonomy and open taxonomy) and demonstrates consistent synergy effects when integrated with previous state-of-the-art UDA methods. The implementation is available at http://github.com/jkee58/CSI.

Related papers

• DynAlign: Unsupervised Dynamic Taxonomy Alignment for Cross-Domain Segmentation [15.303659468173334]
  We introduce DynAlign, a framework that integrates UDA with foundation models to bridge the image-level and label-level domain gaps. Our approach leverages prior semantic knowledge to align source categories with target categories that can be novel, more fine-grained, or named differently. DynAlign generates accurate predictions in a new target label space without requiring any manual annotations.
  arXiv  Detail & Related papers  (2025-01-27T18:57:19Z)
• Taxonomy-Structured Domain Adaptation [21.432546714330023]
  We tackle a generalization with taxonomy-structured domains, which formalizes domains with nested, hierarchical similarity structures. We build on the classic adversarial framework and introduce a novel taxonomist, which competes with the adversarial discriminator to preserve the taxonomy information.
  arXiv  Detail & Related papers  (2023-06-13T16:04:14Z)
• Prototypical Contrast Adaptation for Domain Adaptive Semantic Segmentation [52.63046674453461]
  Prototypical Contrast Adaptation (ProCA) is a contrastive learning method for unsupervised domain adaptive semantic segmentation. ProCA incorporates inter-class information into class-wise prototypes, and adopts the class-centered distribution alignment for adaptation.
  arXiv  Detail & Related papers  (2022-07-14T04:54:26Z)
• Domain Adaptation via Prompt Learning [39.97105851723885]
  Unsupervised domain adaption (UDA) aims to adapt models learned from a well-annotated source domain to a target domain. We introduce a novel prompt learning paradigm for UDA, named Domain Adaptation via Prompt Learning (DAPL)
  arXiv  Detail & Related papers  (2022-02-14T13:25:46Z)
• TADA: Taxonomy Adaptive Domain Adaptation [143.68890984935726]
  Traditional domain adaptation addresses the task of adapting a model to a novel target domain under limited supervision. We introduce the more general taxonomy adaptive domain adaptation problem, allowing for inconsistent between the two domains. On the label-level, we employ a bilateral mixed sampling strategy to augment the target domain, and a relabelling method to unify and align the label spaces.
  arXiv  Detail & Related papers  (2021-09-10T11:58:56Z)
• Contrastive Learning and Self-Training for Unsupervised Domain Adaptation in Semantic Segmentation [71.77083272602525]
  UDA attempts to provide efficient knowledge transfer from a labeled source domain to an unlabeled target domain. We propose a contrastive learning approach that adapts category-wise centroids across domains. We extend our method with self-training, where we use a memory-efficient temporal ensemble to generate consistent and reliable pseudo-labels.
  arXiv  Detail & Related papers  (2021-05-05T11:55:53Z)
• Cluster, Split, Fuse, and Update: Meta-Learning for Open Compound Domain Adaptive Semantic Segmentation [102.42638795864178]
  We propose a principled meta-learning based approach to OCDA for semantic segmentation. We cluster target domain into multiple sub-target domains by image styles, extracted in an unsupervised manner. A meta-learner is thereafter deployed to learn to fuse sub-target domain-specific predictions, conditioned upon the style code. We learn to online update the model by model-agnostic meta-learning (MAML) algorithm, thus to further improve generalization.
  arXiv  Detail & Related papers  (2020-12-15T13:21:54Z)
• Class-Incremental Domain Adaptation [56.72064953133832]
  We introduce a practical Domain Adaptation (DA) paradigm called Class-Incremental Domain Adaptation (CIDA) Existing DA methods tackle domain-shift but are unsuitable for learning novel target-domain classes. Our approach yields superior performance as compared to both DA and CI methods in the CIDA paradigm.
  arXiv  Detail & Related papers  (2020-08-04T07:55:03Z)
• Domain Adaptive Semantic Segmentation Using Weak Labels [115.16029641181669]
  We propose a novel framework for domain adaptation in semantic segmentation with image-level weak labels in the target domain. We develop a weak-label classification module to enforce the network to attend to certain categories. In experiments, we show considerable improvements with respect to the existing state-of-the-arts in UDA and present a new benchmark in the WDA setting.
  arXiv  Detail & Related papers  (2020-07-30T01:33:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.