Federated Semi-Supervised Domain Adaptation via Knowledge Transfer

• URL: http://arxiv.org/abs/2207.10727v1
• Date: Thu, 21 Jul 2022 19:36:10 GMT
• Title: Federated Semi-Supervised Domain Adaptation via Knowledge Transfer
• Authors: Madhureeta Das, Xianhao Chen, Xiaoyong Yuan, Lan Zhang
• Abstract summary: This paper proposes an innovative approach to achieve semi-supervised domain adaptation (SSDA) over multiple distributed and confidential datasets. Federated Semi-Supervised Domain Adaptation (FSSDA) integrates SSDA with federated learning based on strategically designed knowledge distillation techniques. Extensive experiments are conducted to demonstrate the effectiveness and efficiency of FSSDA design.
• Score: 6.7543356061346485
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Given the rapidly changing machine learning environments and expensive data labeling, semi-supervised domain adaptation (SSDA) is imperative when the labeled data from the source domain is statistically different from the partially labeled data from the target domain. Most prior SSDA research is centrally performed, requiring access to both source and target data. However, data in many fields nowadays is generated by distributed end devices. Due to privacy concerns, the data might be locally stored and cannot be shared, resulting in the ineffectiveness of existing SSDA research. This paper proposes an innovative approach to achieve SSDA over multiple distributed and confidential datasets, named by Federated Semi-Supervised Domain Adaptation (FSSDA). FSSDA integrates SSDA with federated learning based on strategically designed knowledge distillation techniques, whose efficiency is improved by performing source and target training in parallel. Moreover, FSSDA controls the amount of knowledge transferred across domains by properly selecting a key parameter, i.e., the imitation parameter. Further, the proposed FSSDA can be effectively generalized to multi-source domain adaptation scenarios. Extensive experiments are conducted to demonstrate the effectiveness and efficiency of FSSDA design.

Related papers

• Semi-Supervised Transfer Boosting (SS-TrBoosting) [13.324712107139355]
  Semi-supervised domain adaptation (SSDA) aims at training a high-performance model for a target domain using few labeled target data. We propose a novel fine-tuning framework, semi-supervised transfer boosting (SS-TrBoosting) For more efficient data transmission and better data privacy protection, we propose a source data generation approach to extend SS-TrBoosting to semi-supervised source-free domain adaptation (SS-SFDA)
  arXiv  Detail & Related papers  (2024-12-04T10:57:55Z)
• Unveiling the Superior Paradigm: A Comparative Study of Source-Free Domain Adaptation and Unsupervised Domain Adaptation [52.36436121884317]
  We show that Source-Free Domain Adaptation (SFDA) generally outperforms Unsupervised Domain Adaptation (UDA) in real-world scenarios. SFDA offers advantages in time efficiency, storage requirements, targeted learning objectives, reduced risk of negative transfer, and increased robustness against overfitting. We propose a novel weight estimation method that effectively integrates available source data into multi-SFDA approaches.
  arXiv  Detail & Related papers  (2024-11-24T13:49:29Z)
• AdaEmbed: Semi-supervised Domain Adaptation in the Embedding Space [11.558794436129192]
  Semi-supervised domain adaptation (SSDA) presents a critical hurdle in computer vision. AdaEmbed facilitates the transfer of knowledge from a labeled source domain to an unlabeled target domain by learning a shared embedding space. Our method's effectiveness is validated through extensive experiments on benchmark datasets such as DomainNet, Office-Home, and VisDA-C.
  arXiv  Detail & Related papers  (2024-01-23T01:10:25Z)
• CoSDA: Continual Source-Free Domain Adaptation [78.47274343972904]
  Without access to the source data, source-free domain adaptation (SFDA) transfers knowledge from a source-domain trained model to target domains. Recently, SFDA has gained popularity due to the need to protect the data privacy of the source domain, but it suffers from catastrophic forgetting on the source domain due to the lack of data. We propose a continual source-free domain adaptation approach named CoSDA, which employs a dual-speed optimized teacher-student model pair and is equipped with consistency learning capability.
  arXiv  Detail & Related papers  (2023-04-13T15:53:23Z)
• Deep Unsupervised Domain Adaptation: A Review of Recent Advances and Perspectives [16.68091981866261]
  Unsupervised domain adaptation (UDA) is proposed to counter the performance drop on data in a target domain. UDA has yielded promising results on natural image processing, video analysis, natural language processing, time-series data analysis, medical image analysis, etc.
  arXiv  Detail & Related papers  (2022-08-15T20:05:07Z)
• Balancing Discriminability and Transferability for Source-Free Domain Adaptation [55.143687986324935]
  Conventional domain adaptation (DA) techniques aim to improve domain transferability by learning domain-invariant representations. The requirement of simultaneous access to labeled source and unlabeled target renders them unsuitable for the challenging source-free DA setting. We derive novel insights to show that a mixup between original and corresponding translated generic samples enhances the discriminability-transferability trade-off.
  arXiv  Detail & Related papers  (2022-06-16T09:06:22Z)
• Source-Free Domain Adaptation via Distribution Estimation [106.48277721860036]
  Domain Adaptation aims to transfer the knowledge learned from a labeled source domain to an unlabeled target domain whose data distributions are different. Recently, Source-Free Domain Adaptation (SFDA) has drawn much attention, which tries to tackle domain adaptation problem without using source data. In this work, we propose a novel framework called SFDA-DE to address SFDA task via source Distribution Estimation.
  arXiv  Detail & Related papers  (2022-04-24T12:22:19Z)
• Automatic Online Multi-Source Domain Adaptation [15.475463516901936]
  An online domain adaptation technique under multisource streaming processes, namely automatic online multi-source domain adaptation (AOMSDA) is proposed in this paper. A numerical study demonstrates that AOMSDA is capable of outperforming its counterparts in 5 of 8 study cases.
  arXiv  Detail & Related papers  (2021-09-05T05:07:16Z)
• Secure Domain Adaptation with Multiple Sources [13.693640425403636]
  Multi-source unsupervised domain adaptation (MUDA) is a recently explored learning framework. The goal is to address the challenge of labeled data scarcity in a target domain via transferring knowledge from multiple source domains with annotated data. Since the source data is distributed, the privacy of source domains' data can be a natural concern. We benefit from the idea of domain alignment in an embedding space to address the privacy concern for MUDA.
  arXiv  Detail & Related papers  (2021-06-23T02:26:36Z)
• Instance Level Affinity-Based Transfer for Unsupervised Domain Adaptation [74.71931918541748]
  We propose an instance affinity based criterion for source to target transfer during adaptation, called ILA-DA. We first propose a reliable and efficient method to extract similar and dissimilar samples across source and target, and utilize a multi-sample contrastive loss to drive the domain alignment process. We verify the effectiveness of ILA-DA by observing consistent improvements in accuracy over popular domain adaptation approaches on a variety of benchmark datasets.
  arXiv  Detail & Related papers  (2021-04-03T01:33:14Z)
• Multi-source Domain Adaptation for Visual Sentiment Classification [92.53780541232773]
  We propose a novel multi-source domain adaptation (MDA) method, termed Multi-source Sentiment Generative Adversarial Network (MSGAN) To handle data from multiple source domains, MSGAN learns to find a unified sentiment latent space where data from both the source and target domains share a similar distribution. Extensive experiments conducted on four benchmark datasets demonstrate that MSGAN significantly outperforms the state-of-the-art MDA approaches for visual sentiment classification.
  arXiv  Detail & Related papers  (2020-01-12T08:37:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.