A Comprehensive Survey of Federated Transfer Learning: Challenges,
Methods and Applications
- URL: http://arxiv.org/abs/2403.01387v1
- Date: Sun, 3 Mar 2024 03:52:27 GMT
- Title: A Comprehensive Survey of Federated Transfer Learning: Challenges,
Methods and Applications
- Authors: Wei Guo, Fuzhen Zhuang, Xiao Zhang, Yiqi Tong, Jin Dong
- Abstract summary: Federated learning (FL) is a distributed machine learning paradigm that enables participants to collaboratively train a centralized model with privacy preservation.
Many FL methods do not work well due to the training and test data of each participant may not be sampled from the same feature space and the same underlying distribution.
To solve this problem, federated transfer learning (FTL) has attracted the attention of numerous researchers.
- Score: 28.677457598856538
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Federated learning (FL) is a novel distributed machine learning paradigm that
enables participants to collaboratively train a centralized model with privacy
preservation by eliminating the requirement of data sharing. In practice, FL
often involves multiple participants and requires the third party to aggregate
global information to guide the update of the target participant. Therefore,
many FL methods do not work well due to the training and test data of each
participant may not be sampled from the same feature space and the same
underlying distribution. Meanwhile, the differences in their local devices
(system heterogeneity), the continuous influx of online data (incremental
data), and labeled data scarcity may further influence the performance of these
methods. To solve this problem, federated transfer learning (FTL), which
integrates transfer learning (TL) into FL, has attracted the attention of
numerous researchers. However, since FL enables a continuous share of knowledge
among participants with each communication round while not allowing local data
to be accessed by other participants, FTL faces many unique challenges that are
not present in TL. In this survey, we focus on categorizing and reviewing the
current progress on federated transfer learning, and outlining corresponding
solutions and applications. Furthermore, the common setting of FTL scenarios,
available datasets, and significant related research are summarized in this
survey.
Related papers
- Federated Large Language Models: Current Progress and Future Directions [63.68614548512534]
This paper surveys Federated learning for LLMs (FedLLM), highlighting recent advances and future directions.
We focus on two key aspects: fine-tuning and prompt learning in a federated setting, discussing existing work and associated research challenges.
arXiv Detail & Related papers (2024-09-24T04:14:33Z) - FedAuxHMTL: Federated Auxiliary Hard-Parameter Sharing Multi-Task Learning for Network Edge Traffic Classification [9.816810723612653]
This paper introduces a new framework for federated auxiliary hard- parameter sharing multi-task learning, namely, FedAuxHMTL.
It incorporates model parameter exchanges between edge server and base stations, enabling base stations from distributed areas to participate in the FedAuxHMTL process.
Empirical experiments are conducted to validate and demonstrate the FedAuxHMTL's effectiveness in terms of accuracy, total global loss, communication costs, computing time, and energy consumption.
arXiv Detail & Related papers (2024-04-11T16:23:28Z) - A review on different techniques used to combat the non-IID and
heterogeneous nature of data in FL [0.0]
Federated Learning (FL) is a machine-learning approach enabling collaborative model training across multiple edge devices.
The significance of FL is particularly pronounced in industries such as healthcare and finance, where data privacy holds paramount importance.
This report delves into the issues arising from non-IID and heterogeneous data and explores current algorithms designed to address these challenges.
arXiv Detail & Related papers (2024-01-01T16:34:00Z) - Integrating Local Real Data with Global Gradient Prototypes for
Classifier Re-Balancing in Federated Long-Tailed Learning [60.41501515192088]
Federated Learning (FL) has become a popular distributed learning paradigm that involves multiple clients training a global model collaboratively.
The data samples usually follow a long-tailed distribution in the real world, and FL on the decentralized and long-tailed data yields a poorly-behaved global model.
In this work, we integrate the local real data with the global gradient prototypes to form the local balanced datasets.
arXiv Detail & Related papers (2023-01-25T03:18:10Z) - Federated Learning and Meta Learning: Approaches, Applications, and
Directions [94.68423258028285]
In this tutorial, we present a comprehensive review of FL, meta learning, and federated meta learning (FedMeta)
Unlike other tutorial papers, our objective is to explore how FL, meta learning, and FedMeta methodologies can be designed, optimized, and evolved, and their applications over wireless networks.
arXiv Detail & Related papers (2022-10-24T10:59:29Z) - Heterogeneous Federated Learning via Grouped Sequential-to-Parallel
Training [60.892342868936865]
Federated learning (FL) is a rapidly growing privacy-preserving collaborative machine learning paradigm.
We propose a data heterogeneous-robust FL approach, FedGSP, to address this challenge.
We show that FedGSP improves the accuracy by 3.7% on average compared with seven state-of-the-art approaches.
arXiv Detail & Related papers (2022-01-31T03:15:28Z) - Privacy-Preserving Self-Taught Federated Learning for Heterogeneous Data [6.545317180430584]
Federated learning (FL) was proposed to enable joint training of a deep learning model using the local data in each party without revealing the data to others.
In this work, we propose an FL method called self-taught federated learning to address the aforementioned issues.
In this method, only latent variables are transmitted to other parties for model training, while privacy is preserved by storing the data and parameters of activations, weights, and biases locally.
arXiv Detail & Related papers (2021-02-11T08:07:51Z) - FedH2L: Federated Learning with Model and Statistical Heterogeneity [75.61234545520611]
Federated learning (FL) enables distributed participants to collectively learn a strong global model without sacrificing their individual data privacy.
We introduce FedH2L, which is agnostic to both the model architecture and robust to different data distributions across participants.
In contrast to approaches sharing parameters or gradients, FedH2L relies on mutual distillation, exchanging only posteriors on a shared seed set between participants in a decentralized manner.
arXiv Detail & Related papers (2021-01-27T10:10:18Z) - Mitigating Bias in Federated Learning [9.295028968787351]
In this paper, we discuss causes of bias in federated learning (FL)
We propose three pre-processing and in-processing methods to mitigate bias, without compromising data privacy.
We conduct experiments over several data distributions to analyze their effects on model performance, fairness metrics, and bias learning patterns.
arXiv Detail & Related papers (2020-12-04T08:04:12Z) - A Principled Approach to Data Valuation for Federated Learning [73.19984041333599]
Federated learning (FL) is a popular technique to train machine learning (ML) models on decentralized data sources.
The Shapley value (SV) defines a unique payoff scheme that satisfies many desiderata for a data value notion.
This paper proposes a variant of the SV amenable to FL, which we call the federated Shapley value.
arXiv Detail & Related papers (2020-09-14T04:37:54Z) - Semi-Federated Learning [4.897245729430157]
Federated learning (FL) enables massive distributed Information and Communication Technology (ICT) devices to learn a global consensus model without any participants revealing their own data to the central server.
We propose the Semi-Federated Learning (Semi-FL) which differs from the FL in two aspects, local clients clustering and in-cluster training.
The proposed Semi-FL can be easily applied to future mobile communication networks and require less up-link transmission bandwidth.
arXiv Detail & Related papers (2020-03-28T14:08:59Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.