Multi-Source to Multi-Target Decentralized Federated Domain Adaptation
- URL: http://arxiv.org/abs/2304.12422v2
- Date: Tue, 9 Jan 2024 03:37:06 GMT
- Title: Multi-Source to Multi-Target Decentralized Federated Domain Adaptation
- Authors: Su Wang, Seyyedali Hosseinalipour, Christopher G. Brinton
- Abstract summary: In this paper, we focus on varying quantities/distributions of labeled and unlabeled data across devices.
We develop a decentralized federated domain adaptation methodology which considers the transfer of ML models from devices with high quality labeled data to devices with low quality or unlabeled data.
Our methodology, Source-Target Determination and Link Formation (ST-LF), optimize both (i) classification of devices into sources and targets and (ii) source-target link formation.
- Score: 15.681197161658835
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Heterogeneity across devices in federated learning (FL) typically refers to
statistical (e.g., non-i.i.d. data distributions) and resource (e.g.,
communication bandwidth) dimensions. In this paper, we focus on another
important dimension that has received less attention: varying
quantities/distributions of labeled and unlabeled data across devices. In order
to leverage all data, we develop a decentralized federated domain adaptation
methodology which considers the transfer of ML models from devices with high
quality labeled data (called sources) to devices with low quality or unlabeled
data (called targets). Our methodology, Source-Target Determination and Link
Formation (ST-LF), optimizes both (i) classification of devices into sources
and targets and (ii) source-target link formation, in a manner that considers
the trade-off between ML model accuracy and communication energy efficiency. To
obtain a concrete objective function, we derive a measurable generalization
error bound that accounts for estimates of source-target hypothesis deviations
and divergences between data distributions. The resulting optimization problem
is a mixed-integer signomial program, a class of NP-hard problems, for which we
develop an algorithm based on successive convex approximations to solve it
tractably. Subsequent numerical evaluations of ST-LF demonstrate that it
improves classification accuracy and energy efficiency over state-of-the-art
baselines.
Related papers
- Filling the Missing: Exploring Generative AI for Enhanced Federated
Learning over Heterogeneous Mobile Edge Devices [72.61177465035031]
We propose a generative AI-empowered federated learning to address these challenges by leveraging the idea of FIlling the MIssing (FIMI) portion of local data.
Experiment results demonstrate that FIMI can save up to 50% of the device-side energy to achieve the target global test accuracy.
arXiv Detail & Related papers (2023-10-21T12:07:04Z) - Analysis and Optimization of Wireless Federated Learning with Data
Heterogeneity [72.85248553787538]
This paper focuses on performance analysis and optimization for wireless FL, considering data heterogeneity, combined with wireless resource allocation.
We formulate the loss function minimization problem, under constraints on long-term energy consumption and latency, and jointly optimize client scheduling, resource allocation, and the number of local training epochs (CRE)
Experiments on real-world datasets demonstrate that the proposed algorithm outperforms other benchmarks in terms of the learning accuracy and energy consumption.
arXiv Detail & Related papers (2023-08-04T04:18:01Z) - Domain Adaptation Principal Component Analysis: base linear method for
learning with out-of-distribution data [55.41644538483948]
Domain adaptation is a popular paradigm in modern machine learning.
We present a method called Domain Adaptation Principal Component Analysis (DAPCA)
DAPCA finds a linear reduced data representation useful for solving the domain adaptation task.
arXiv Detail & Related papers (2022-08-28T21:10:56Z) - Source-Free Domain Adaptation via Distribution Estimation [106.48277721860036]
Domain Adaptation aims to transfer the knowledge learned from a labeled source domain to an unlabeled target domain whose data distributions are different.
Recently, Source-Free Domain Adaptation (SFDA) has drawn much attention, which tries to tackle domain adaptation problem without using source data.
In this work, we propose a novel framework called SFDA-DE to address SFDA task via source Distribution Estimation.
arXiv Detail & Related papers (2022-04-24T12:22:19Z) - FedADMM: A Robust Federated Deep Learning Framework with Adaptivity to
System Heterogeneity [4.2059108111562935]
Federated Learning (FL) is an emerging framework for distributed processing of large data volumes by edge devices.
In this paper, we introduce a new FLAD FedADMM based protocol.
We show that FedADMM consistently outperforms all baseline methods in terms of communication efficiency.
arXiv Detail & Related papers (2022-04-07T15:58:33Z) - A Newton-type algorithm for federated learning based on incremental
Hessian eigenvector sharing [5.404315085380945]
We present an original communication-constrained Newton-type (NT) algorithm designed to accelerate Federated Learning (FL)
The proposed solution is thoroughly validated on real datasets.
arXiv Detail & Related papers (2022-02-11T17:52:56Z) - Resource-constrained Federated Edge Learning with Heterogeneous Data:
Formulation and Analysis [8.863089484787835]
We propose a distributed approximate Newton-type Newton-type training scheme, namely FedOVA, to solve the heterogeneous statistical challenge brought by heterogeneous data.
FedOVA decomposes a multi-class classification problem into more straightforward binary classification problems and then combines their respective outputs using ensemble learning.
arXiv Detail & Related papers (2021-10-14T17:35:24Z) - Transfer Learning under High-dimensional Generalized Linear Models [7.675822266933702]
We study the transfer learning problem under high-dimensional generalized linear models.
We propose an oracle algorithm and derive its $ell$-estimation error bounds.
When we don't know which sources to transfer, an algorithm-free transferable source detection approach is introduced.
arXiv Detail & Related papers (2021-05-29T15:39:43Z) - Instance Level Affinity-Based Transfer for Unsupervised Domain
Adaptation [74.71931918541748]
We propose an instance affinity based criterion for source to target transfer during adaptation, called ILA-DA.
We first propose a reliable and efficient method to extract similar and dissimilar samples across source and target, and utilize a multi-sample contrastive loss to drive the domain alignment process.
We verify the effectiveness of ILA-DA by observing consistent improvements in accuracy over popular domain adaptation approaches on a variety of benchmark datasets.
arXiv Detail & Related papers (2021-04-03T01:33:14Z) - Do We Really Need to Access the Source Data? Source Hypothesis Transfer
for Unsupervised Domain Adaptation [102.67010690592011]
Unsupervised adaptationUDA (UDA) aims to leverage the knowledge learned from a labeled source dataset to solve similar tasks in a new unlabeled domain.
Prior UDA methods typically require to access the source data when learning to adapt the model.
This work tackles a practical setting where only a trained source model is available and how we can effectively utilize such a model without source data to solve UDA problems.
arXiv Detail & Related papers (2020-02-20T03:13:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.