Progressive Modality Cooperation for Multi-Modality Domain Adaptation

• URL: http://arxiv.org/abs/2506.19316v1
• Date: Tue, 24 Jun 2025 05:13:56 GMT
• Title: Progressive Modality Cooperation for Multi-Modality Domain Adaptation
• Authors: Weichen Zhang, Dong Xu, Jing Zhang, Wanli Ouyang,
• Abstract summary: We propose a new generic multi-modality domain adaptation framework called Progressive Modality Cooperation (PMC)<n>Under the MMDA setting, the samples in both domains have all the modalities.<n>Under the MMDA-PI setting, some modalities are missing in the target domain.
• Score: 70.26879294371052
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work, we propose a new generic multi-modality domain adaptation framework called Progressive Modality Cooperation (PMC) to transfer the knowledge learned from the source domain to the target domain by exploiting multiple modality clues (\eg, RGB and depth) under the multi-modality domain adaptation (MMDA) and the more general multi-modality domain adaptation using privileged information (MMDA-PI) settings. Under the MMDA setting, the samples in both domains have all the modalities. In two newly proposed modules of our PMC, the multiple modalities are cooperated for selecting the reliable pseudo-labeled target samples, which captures the modality-specific information and modality-integrated information, respectively. Under the MMDA-PI setting, some modalities are missing in the target domain. Hence, to better exploit the multi-modality data in the source domain, we further propose the PMC with privileged information (PMC-PI) method by proposing a new multi-modality data generation (MMG) network. MMG generates the missing modalities in the target domain based on the source domain data by considering both domain distribution mismatch and semantics preservation, which are respectively achieved by using adversarial learning and conditioning on weighted pseudo semantics. Extensive experiments on three image datasets and eight video datasets for various multi-modality cross-domain visual recognition tasks under both MMDA and MMDA-PI settings clearly demonstrate the effectiveness of our proposed PMC framework.

Related papers

• Collaborative Multi-source Domain Adaptation Through Optimal Transport [0.0]
  Multi-source Domain Adaptation (MDA) seeks to adapt models trained on data from multiple labeled source domains to perform effectively on an unlabeled target domain data. We introduce Collaborative MDA Through Optimal Transport (CMDA-OT), a novel framework consisting of two key phases.
  arXiv  Detail & Related papers  (2024-04-09T20:06:25Z)
• Multi-Prompt Alignment for Multi-Source Unsupervised Domain Adaptation [86.02485817444216]
  We introduce Multi-Prompt Alignment (MPA), a simple yet efficient framework for multi-source UDA. MPA denoises the learned prompts through an auto-encoding process and aligns them by maximizing the agreement of all the reconstructed prompts. Experiments show that MPA achieves state-of-the-art results on three popular datasets with an impressive average accuracy of 54.1% on DomainNet.
  arXiv  Detail & Related papers  (2022-09-30T03:40:10Z)
• Multi-Source Unsupervised Domain Adaptation via Pseudo Target Domain [0.0]
  Multi-source domain adaptation (MDA) aims to transfer knowledge from multiple source domains to an unlabeled target domain. We propose a novel MDA approach, termed Pseudo Target for MDA (PTMDA) PTMDA maps each group of source and target domains into a group-specific subspace using adversarial learning with a metric constraint. We show that PTMDA as a whole can reduce the target error bound and leads to a better approximation of the target risk in MDA settings.
  arXiv  Detail & Related papers  (2022-02-22T08:37:16Z)
• Graphical Modeling for Multi-Source Domain Adaptation [56.05348879528149]
  Multi-Source Domain Adaptation (MSDA) focuses on transferring the knowledge from multiple source domains to the target domain. We propose two types of graphical models,i.e. Conditional Random Field for MSDA (CRF-MSDA) and Markov Random Field for MSDA (MRF-MSDA) We evaluate these two models on four standard benchmark data sets of MSDA with distinct domain shift and data complexity.
  arXiv  Detail & Related papers  (2021-04-27T09:04:22Z)
• Cluster, Split, Fuse, and Update: Meta-Learning for Open Compound Domain Adaptive Semantic Segmentation [102.42638795864178]
  We propose a principled meta-learning based approach to OCDA for semantic segmentation. We cluster target domain into multiple sub-target domains by image styles, extracted in an unsupervised manner. A meta-learner is thereafter deployed to learn to fuse sub-target domain-specific predictions, conditioned upon the style code. We learn to online update the model by model-agnostic meta-learning (MAML) algorithm, thus to further improve generalization.
  arXiv  Detail & Related papers  (2020-12-15T13:21:54Z)
• Mutual Learning Network for Multi-Source Domain Adaptation [73.25974539191553]
  We propose a novel multi-source domain adaptation method, Mutual Learning Network for Multiple Source Domain Adaptation (ML-MSDA) Under the framework of mutual learning, the proposed method pairs the target domain with each single source domain to train a conditional adversarial domain adaptation network as a branch network. The proposed method outperforms the comparison methods and achieves the state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-03-29T04:31:43Z)
• Unified Multi-Domain Learning and Data Imputation using Adversarial Autoencoder [5.933303832684138]
  We present a novel framework that can combine multi-domain learning (MDL), data imputation (DI) and multi-task learning (MTL) The core of our method is an adversarial autoencoder that can: (1) learn to produce domain-invariant embeddings to reduce the difference between domains; (2) learn the data distribution for each domain and correctly perform data imputation on missing data.
  arXiv  Detail & Related papers  (2020-03-15T19:55:07Z)
• MADAN: Multi-source Adversarial Domain Aggregation Network for Domain Adaptation [58.38749495295393]
  Domain adaptation aims to learn a transferable model to bridge the domain shift between one labeled source domain and another sparsely labeled or unlabeled target domain. Recent multi-source domain adaptation (MDA) methods do not consider the pixel-level alignment between sources and target. We propose a novel MDA framework to address these challenges.
  arXiv  Detail & Related papers  (2020-02-19T21:22:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.