DCFL: Non-IID awareness Data Condensation aided Federated Learning
- URL: http://arxiv.org/abs/2312.14219v1
- Date: Thu, 21 Dec 2023 13:04:24 GMT
- Title: DCFL: Non-IID awareness Data Condensation aided Federated Learning
- Authors: Shaohan Sha and YaFeng Sun
- Abstract summary: Federated learning is a decentralized learning paradigm wherein a central server trains a global model iteratively by utilizing clients who possess a certain amount of private datasets.
The challenge lies in the fact that the client side private data may not be identically and independently distributed.
We propose DCFL which divides clients into groups by using the Centered Kernel Alignment (CKA) method, then uses dataset condensation methods with non-IID awareness to complete clients.
- Score: 0.8158530638728501
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Federated learning is a decentralized learning paradigm wherein a central
server trains a global model iteratively by utilizing clients who possess a
certain amount of private datasets. The challenge lies in the fact that the
client side private data may not be identically and independently distributed,
significantly impacting the accuracy of the global model. Existing methods
commonly address the Non-IID challenge by focusing on optimization, client
selection and data complement. However, most approaches tend to overlook the
perspective of the private data itself due to privacy constraints.Intuitively,
statistical distinctions among private data on the client side can help
mitigate the Non-IID degree. Besides, the recent advancements in dataset
condensation technology have inspired us to investigate its potential
applicability in addressing Non-IID issues while maintaining privacy. Motivated
by this, we propose DCFL which divides clients into groups by using the
Centered Kernel Alignment (CKA) method, then uses dataset condensation methods
with non-IID awareness to complete clients. The private data from clients
within the same group is complementary and their condensed data is accessible
to all clients in the group. Additionally, CKA-guided client selection
strategy, filtering mechanisms, and data enhancement techniques are
incorporated to efficiently and precisely utilize the condensed data, enhance
model performance, and minimize communication time. Experimental results
demonstrate that DCFL achieves competitive performance on popular federated
learning benchmarks including MNIST, FashionMNIST, SVHN, and CIFAR-10 with
existing FL protocol.
Related papers
- An Aggregation-Free Federated Learning for Tackling Data Heterogeneity [50.44021981013037]
Federated Learning (FL) relies on the effectiveness of utilizing knowledge from distributed datasets.
Traditional FL methods adopt an aggregate-then-adapt framework, where clients update local models based on a global model aggregated by the server from the previous training round.
We introduce FedAF, a novel aggregation-free FL algorithm.
arXiv Detail & Related papers (2024-04-29T05:55:23Z) - Personalized Federated Learning with Attention-based Client Selection [57.71009302168411]
We propose FedACS, a new PFL algorithm with an Attention-based Client Selection mechanism.
FedACS integrates an attention mechanism to enhance collaboration among clients with similar data distributions.
Experiments on CIFAR10 and FMNIST validate FedACS's superiority.
arXiv Detail & Related papers (2023-12-23T03:31:46Z) - Personalized Privacy-Preserving Framework for Cross-Silo Federated
Learning [0.0]
Federated learning (FL) is a promising decentralized deep learning (DL) framework that enables DL-based approaches trained collaboratively across clients without sharing private data.
In this paper, we propose a novel framework, namely Personalized Privacy-Preserving Federated Learning (PPPFL)
Our proposed framework outperforms multiple FL baselines on different datasets, including MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100.
arXiv Detail & Related papers (2023-02-22T07:24:08Z) - CADIS: Handling Cluster-skewed Non-IID Data in Federated Learning with
Clustered Aggregation and Knowledge DIStilled Regularization [3.3711670942444014]
Federated learning enables edge devices to train a global model collaboratively without exposing their data.
We tackle a new type of Non-IID data, called cluster-skewed non-IID, discovered in actual data sets.
We propose an aggregation scheme that guarantees equality between clusters.
arXiv Detail & Related papers (2023-02-21T02:53:37Z) - Knowledge-Aware Federated Active Learning with Non-IID Data [75.98707107158175]
We propose a federated active learning paradigm to efficiently learn a global model with limited annotation budget.
The main challenge faced by federated active learning is the mismatch between the active sampling goal of the global model on the server and that of the local clients.
We propose Knowledge-Aware Federated Active Learning (KAFAL), which consists of Knowledge-Specialized Active Sampling (KSAS) and Knowledge-Compensatory Federated Update (KCFU)
arXiv Detail & Related papers (2022-11-24T13:08:43Z) - Rethinking Data Heterogeneity in Federated Learning: Introducing a New
Notion and Standard Benchmarks [65.34113135080105]
We show that not only the issue of data heterogeneity in current setups is not necessarily a problem but also in fact it can be beneficial for the FL participants.
Our observations are intuitive.
Our code is available at https://github.com/MMorafah/FL-SC-NIID.
arXiv Detail & Related papers (2022-09-30T17:15:19Z) - Federated Learning in Non-IID Settings Aided by Differentially Private
Synthetic Data [20.757477553095637]
Federated learning (FL) is a privacy-promoting framework that enables clients to collaboratively train machine learning models.
A major challenge in federated learning arises when the local data is heterogeneous.
We propose FedDPMS, an FL algorithm in which clients deploy variational auto-encoders to augment local datasets with data synthesized using differentially private means of latent data representations.
arXiv Detail & Related papers (2022-06-01T18:00:48Z) - FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling
and Correction [48.85303253333453]
Federated learning (FL) allows multiple clients to collectively train a high-performance global model without sharing their private data.
We propose a novel federated learning algorithm with local drift decoupling and correction (FedDC)
Our FedDC only introduces lightweight modifications in the local training phase, in which each client utilizes an auxiliary local drift variable to track the gap between the local model parameter and the global model parameters.
Experiment results and analysis demonstrate that FedDC yields expediting convergence and better performance on various image classification tasks.
arXiv Detail & Related papers (2022-03-22T14:06:26Z) - ABC-FL: Anomalous and Benign client Classification in Federated Learning [0.0]
Federated Learning is a distributed machine learning framework designed for data privacy preservation.
It inherits the vulnerabilities and susceptibilities raised in deep learning techniques.
It is difficult to correctly identify malicious clients due to the non-Independently and/or Identically Distributed (non-IID) data.
We propose a method that detects and classifies anomalous clients from benign clients when benign ones have non-IID data.
arXiv Detail & Related papers (2021-08-10T09:54:25Z) - IFedAvg: Interpretable Data-Interoperability for Federated Learning [39.388223565330385]
In this work, we define and address low interoperability induced by underlying client data inconsistencies in federated learning for tabular data.
The proposed method, iFedAvg, builds on federated averaging adding local element-wise affine layers to allow for a personalized and granular understanding of the collaborative learning process.
We evaluate iFedAvg using several public benchmarks and a collection of real-world datasets from the 2014 - 2016 West African Ebola epidemic, jointly forming the largest such dataset in the world.
arXiv Detail & Related papers (2021-07-14T09:54:00Z) - Towards Fair Federated Learning with Zero-Shot Data Augmentation [123.37082242750866]
Federated learning has emerged as an important distributed learning paradigm, where a server aggregates a global model from many client-trained models while having no access to the client data.
We propose a novel federated learning system that employs zero-shot data augmentation on under-represented data to mitigate statistical heterogeneity and encourage more uniform accuracy performance across clients in federated networks.
We study two variants of this scheme, Fed-ZDAC (federated learning with zero-shot data augmentation at the clients) and Fed-ZDAS (federated learning with zero-shot data augmentation at the server).
arXiv Detail & Related papers (2021-04-27T18:23:54Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.