Towards Federated Long-Tailed Learning
- URL: http://arxiv.org/abs/2206.14988v1
- Date: Thu, 30 Jun 2022 02:34:22 GMT
- Title: Towards Federated Long-Tailed Learning
- Authors: Zihan Chen, Songshang Liu, Hualiang Wang, Howard H. Yang, Tony Q.S.
Quek and Zuozhu Liu
- Abstract summary: Data privacy and class imbalance are the norm rather than the exception in many machine learning tasks.
Recent attempts have been launched to, on one side, address the problem of learning from pervasive private data, and on the other side, learn from long-tailed data.
This paper focuses on learning with long-tailed (LT) data distributions under the context of the popular privacy-preserved federated learning (FL) framework.
- Score: 76.50892783088702
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Data privacy and class imbalance are the norm rather than the exception in
many machine learning tasks. Recent attempts have been launched to, on one
side, address the problem of learning from pervasive private data, and on the
other side, learn from long-tailed data. However, both assumptions might hold
in practical applications, while an effective method to simultaneously
alleviate both issues is yet under development. In this paper, we focus on
learning with long-tailed (LT) data distributions under the context of the
popular privacy-preserved federated learning (FL) framework. We characterize
three scenarios with different local or global long-tailed data distributions
in the FL framework, and highlight the corresponding challenges. The
preliminary results under different scenarios reveal that substantial future
work are of high necessity to better resolve the characterized federated
long-tailed learning tasks.
Related papers
- Federated Large Language Models: Current Progress and Future Directions [63.68614548512534]
This paper surveys Federated learning for LLMs (FedLLM), highlighting recent advances and future directions.
We focus on two key aspects: fine-tuning and prompt learning in a federated setting, discussing existing work and associated research challenges.
arXiv Detail & Related papers (2024-09-24T04:14:33Z) - A Survey of Deep Long-Tail Classification Advancements [1.6233132273470656]
Many data distributions in the real world are hardly uniform. Instead, skewed and long-tailed distributions of various kinds are commonly observed.
This poses an interesting problem for machine learning, where most algorithms assume or work well with uniformly distributed data.
The problem is further exacerbated by current state-of-the-art deep learning models requiring large volumes of training data.
arXiv Detail & Related papers (2024-04-24T01:59:02Z) - Federated Learning Empowered by Generative Content [55.576885852501775]
Federated learning (FL) enables leveraging distributed private data for model training in a privacy-preserving way.
We propose a novel FL framework termed FedGC, designed to mitigate data heterogeneity issues by diversifying private data with generative content.
We conduct a systematic empirical study on FedGC, covering diverse baselines, datasets, scenarios, and modalities.
arXiv Detail & Related papers (2023-12-10T07:38:56Z) - A Survey of Federated Unlearning: A Taxonomy, Challenges and Future
Directions [71.16718184611673]
The evolution of privacy-preserving Federated Learning (FL) has led to an increasing demand for implementing the right to be forgotten.
The implementation of selective forgetting is particularly challenging in FL due to its decentralized nature.
Federated Unlearning (FU) emerges as a strategic solution to address the increasing need for data privacy.
arXiv Detail & Related papers (2023-10-30T01:34:33Z) - A Survey on Class Imbalance in Federated Learning [6.632451878730774]
Federated learning allows multiple client devices in a network to jointly train a machine learning model without direct exposure of clients' data.
It has been found that models trained with federated learning usually have worse performance than their counterparts trained in the standard centralized learning mode.
arXiv Detail & Related papers (2023-03-21T08:34:23Z) - FedOS: using open-set learning to stabilize training in federated
learning [0.0]
Federated Learning is a new approach to train statistical models on distributed datasets without violating privacy constraints.
This report explores this new research area and performs several experiments to deepen our understanding of what these challenges are.
We present a novel approach to one of these challenges and compare it to other methods found in literature.
arXiv Detail & Related papers (2022-08-22T19:53:39Z) - FEDIC: Federated Learning on Non-IID and Long-Tailed Data via Calibrated
Distillation [54.2658887073461]
Dealing with non-IID data is one of the most challenging problems for federated learning.
This paper studies the joint problem of non-IID and long-tailed data in federated learning and proposes a corresponding solution called Federated Ensemble Distillation with Imbalance (FEDIC)
FEDIC uses model ensemble to take advantage of the diversity of models trained on non-IID data.
arXiv Detail & Related papers (2022-04-30T06:17:36Z) - Non-IID data and Continual Learning processes in Federated Learning: A
long road ahead [58.720142291102135]
Federated Learning is a novel framework that allows multiple devices or institutions to train a machine learning model collaboratively while preserving their data private.
In this work, we formally classify data statistical heterogeneity and review the most remarkable learning strategies that are able to face it.
At the same time, we introduce approaches from other machine learning frameworks, such as Continual Learning, that also deal with data heterogeneity and could be easily adapted to the Federated Learning settings.
arXiv Detail & Related papers (2021-11-26T09:57:11Z) - Federated Learning on Non-IID Data Silos: An Experimental Study [34.28108345251376]
Training data have been increasingly fragmented, forming distributed databases of multiple data silos.
In this paper, we propose comprehensive data partitioning strategies to cover the typical non-IID data cases.
We find that non-IID does bring significant challenges in learning accuracy of FL algorithms, and none of the existing state-of-the-art FL algorithms outperforms others in all cases.
arXiv Detail & Related papers (2021-02-03T14:29:09Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.