Robust Aggregation for Federated Learning
- URL: http://arxiv.org/abs/1912.13445v2
- Date: Mon, 17 Jan 2022 05:25:59 GMT
- Title: Robust Aggregation for Federated Learning
- Authors: Krishna Pillutla, Sham M. Kakade, Zaid Harchaoui
- Abstract summary: Federated learning is the centralized training of statistical models from decentralized data on mobile devices.
We present a robust aggregation approach to make federated learning robust to settings when a fraction of the devices may be sending corrupted updates to the server.
- Score: 37.47208810846432
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Federated learning is the centralized training of statistical models from
decentralized data on mobile devices while preserving the privacy of each
device. We present a robust aggregation approach to make federated learning
robust to settings when a fraction of the devices may be sending corrupted
updates to the server. The approach relies on a robust aggregation oracle based
on the geometric median, which returns a robust aggregate using a constant
number of iterations of a regular non-robust averaging oracle. The robust
aggregation oracle is privacy-preserving, similar to the non-robust secure
average oracle it builds upon. We establish its convergence for least squares
estimation of additive models. We provide experimental results with linear
models and deep networks for three tasks in computer vision and natural
language processing. The robust aggregation approach is agnostic to the level
of corruption; it outperforms the classical aggregation approach in terms of
robustness when the level of corruption is high, while being competitive in the
regime of low corruption. Two variants, a faster one with one-step robust
aggregation and another one with on-device personalization, round off the
paper.
Related papers
- PriRoAgg: Achieving Robust Model Aggregation with Minimum Privacy Leakage for Federated Learning [49.916365792036636]
Federated learning (FL) has recently gained significant momentum due to its potential to leverage large-scale distributed user data.
The transmitted model updates can potentially leak sensitive user information, and the lack of central control of the local training process leaves the global model susceptible to malicious manipulations on model updates.
We develop a general framework PriRoAgg, utilizing Lagrange coded computing and distributed zero-knowledge proof, to execute a wide range of robust aggregation algorithms while satisfying aggregated privacy.
arXiv Detail & Related papers (2024-07-12T03:18:08Z) - FedMPQ: Secure and Communication-Efficient Federated Learning with Multi-codebook Product Quantization [12.83265009728818]
We propose a novel uplink communication compression method for federated learning, named FedMPQ.
In contrast to previous works, our approach exhibits greater robustness in scenarios where data is not independently and identically distributed.
Experiments conducted on the LEAF dataset demonstrate that our proposed method achieves 99% of the baseline's final accuracy.
arXiv Detail & Related papers (2024-04-21T08:27:36Z) - Near-Optimal Resilient Aggregation Rules for Distributed Learning Using 1-Center and 1-Mean Clustering with Outliers [24.88026399458157]
Byzantine machine learning has garnered considerable attention in light of the unpredictable faults that can occur.
The key to secure machines in distributed learning is resilient aggregation mechanisms.
arXiv Detail & Related papers (2023-12-20T08:36:55Z) - BRFL: A Blockchain-based Byzantine-Robust Federated Learning Model [8.19957400564017]
Federated learning, which stores data in distributed nodes and shares only model parameters, has gained significant attention for addressing this concern.
A challenge arises in federated learning due to the Byzantine Attack Problem, where malicious local models can compromise the global model's performance during aggregation.
This article proposes the integration of Byzantine-Robust Federated Learning (BRLF) model that combines federated learning with blockchain technology.
arXiv Detail & Related papers (2023-10-20T10:21:50Z) - FedSkip: Combatting Statistical Heterogeneity with Federated Skip
Aggregation [95.85026305874824]
We introduce a data-driven approach called FedSkip to improve the client optima by periodically skipping federated averaging and scattering local models to the cross devices.
We conduct extensive experiments on a range of datasets to demonstrate that FedSkip achieves much higher accuracy, better aggregation efficiency and competing communication efficiency.
arXiv Detail & Related papers (2022-12-14T13:57:01Z) - FedHiSyn: A Hierarchical Synchronous Federated Learning Framework for
Resource and Data Heterogeneity [56.82825745165945]
Federated Learning (FL) enables training a global model without sharing the decentralized raw data stored on multiple devices to protect data privacy.
We propose a hierarchical synchronous FL framework, i.e., FedHiSyn, to tackle the problems of straggler effects and outdated models.
We evaluate the proposed framework based on MNIST, EMNIST, CIFAR10 and CIFAR100 datasets and diverse heterogeneous settings of devices.
arXiv Detail & Related papers (2022-06-21T17:23:06Z) - Performance Weighting for Robust Federated Learning Against Corrupted
Sources [1.76179873429447]
Federated learning has emerged as a dominant computational paradigm for distributed machine learning.
In real-world applications, a federated environment may consist of a mixture of benevolent and malicious clients.
We show that the standard global aggregation scheme of local weights is inefficient in the presence of corrupted clients.
arXiv Detail & Related papers (2022-05-02T20:01:44Z) - Meta Clustering Learning for Large-scale Unsupervised Person
Re-identification [124.54749810371986]
We propose a "small data for big task" paradigm dubbed Meta Clustering Learning (MCL)
MCL only pseudo-labels a subset of the entire unlabeled data via clustering to save computing for the first-phase training.
Our method significantly saves computational cost while achieving a comparable or even better performance compared to prior works.
arXiv Detail & Related papers (2021-11-19T04:10:18Z) - Coded Stochastic ADMM for Decentralized Consensus Optimization with Edge
Computing [113.52575069030192]
Big data, including applications with high security requirements, are often collected and stored on multiple heterogeneous devices, such as mobile devices, drones and vehicles.
Due to the limitations of communication costs and security requirements, it is of paramount importance to extract information in a decentralized manner instead of aggregating data to a fusion center.
We consider the problem of learning model parameters in a multi-agent system with data locally processed via distributed edge nodes.
A class of mini-batch alternating direction method of multipliers (ADMM) algorithms is explored to develop the distributed learning model.
arXiv Detail & Related papers (2020-10-02T10:41:59Z) - Device Heterogeneity in Federated Learning: A Superquantile Approach [0.0]
We propose a framework to handle heterogeneous client devices which do not conform to the population data distribution.
We present an optimization algorithm and establish its convergence to a stationary point.
We conclude with numerical experiments on neural networks as well as linear models on tasks from computer vision and natural language processing.
arXiv Detail & Related papers (2020-02-25T23:37:35Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.