Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning

• URL: http://arxiv.org/abs/2002.04156v3
• Date: Sat, 20 Feb 2021 20:20:49 GMT
• Title: Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning
• Authors: Jinhyun So, Basak Guler, and A. Salman Avestimehr
• Abstract summary: A major bottleneck in scaling federated learning to a large number of users is the overhead of secure model aggregation across many users. In this paper, we propose the first secure aggregation framework, named Turbo-Aggregate, that achieves a secure aggregation overhead of $O(NlogN)$. We experimentally demonstrate that Turbo-Aggregate achieves a total running time that grows almost linear in the number of users, and provides up to $40times$ speedup over the state-of-the-art protocols with up to $N=200$ users.
• Score: 2.294014185517203
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated learning is a distributed framework for training machine learning models over the data residing at mobile devices, while protecting the privacy of individual users. A major bottleneck in scaling federated learning to a large number of users is the overhead of secure model aggregation across many users. In particular, the overhead of the state-of-the-art protocols for secure model aggregation grows quadratically with the number of users. In this paper, we propose the first secure aggregation framework, named Turbo-Aggregate, that in a network with $N$ users achieves a secure aggregation overhead of $O(N\log{N})$, as opposed to $O(N^2)$, while tolerating up to a user dropout rate of $50\%$. Turbo-Aggregate employs a multi-group circular strategy for efficient model aggregation, and leverages additive secret sharing and novel coding techniques for injecting aggregation redundancy in order to handle user dropouts while guaranteeing user privacy. We experimentally demonstrate that Turbo-Aggregate achieves a total running time that grows almost linear in the number of users, and provides up to $40\times$ speedup over the state-of-the-art protocols with up to $N=200$ users. Our experiments also demonstrate the impact of model size and bandwidth on the performance of Turbo-Aggregate.

Related papers

• $\mathsf{OPA}$: One-shot Private Aggregation with Single Client Interaction and its Applications to Federated Learning [6.977111770337479]
  We introduce One-shot Private Aggregation ($mathsfOPA$) where clients speak only once (or even choose not to speak) per aggregation evaluation. Since each client communicates only once per aggregation, this simplifies managing dropouts and dynamic participation. $mathsfOPA$ is practical, outperforming state-of-the-art solutions.
  arXiv  Detail & Related papers  (2024-10-29T17:50:11Z)
• PriRoAgg: Achieving Robust Model Aggregation with Minimum Privacy Leakage for Federated Learning [49.916365792036636]
  Federated learning (FL) has recently gained significant momentum due to its potential to leverage large-scale distributed user data. The transmitted model updates can potentially leak sensitive user information, and the lack of central control of the local training process leaves the global model susceptible to malicious manipulations on model updates. We develop a general framework PriRoAgg, utilizing Lagrange coded computing and distributed zero-knowledge proof, to execute a wide range of robust aggregation algorithms while satisfying aggregated privacy.
  arXiv  Detail & Related papers  (2024-07-12T03:18:08Z)
• Scalable Collaborative Learning via Representation Sharing [53.047460465980144]
  Federated learning (FL) and Split Learning (SL) are two frameworks that enable collaborative learning while keeping the data private (on device) In FL, each data holder trains a model locally and releases it to a central server for aggregation. In SL, the clients must release individual cut-layer activations (smashed data) to the server and wait for its response (during both inference and back propagation). In this work, we present a novel approach for privacy-preserving machine learning, where the clients collaborate via online knowledge distillation using a contrastive loss.
  arXiv  Detail & Related papers  (2022-11-20T10:49:22Z)
• Optimizing Server-side Aggregation For Robust Federated Learning via Subspace Training [80.03567604524268]
  Non-IID data distribution across clients and poisoning attacks are two main challenges in real-world federated learning systems. We propose SmartFL, a generic approach that optimize the server-side aggregation process. We provide theoretical analyses of the convergence and generalization capacity for SmartFL.
  arXiv  Detail & Related papers  (2022-11-10T13:20:56Z)
• Robust Quantity-Aware Aggregation for Federated Learning [72.59915691824624]
  Malicious clients can poison model updates and claim large quantities to amplify the impact of their model updates in the model aggregation. Existing defense methods for FL, while all handling malicious model updates, either treat all quantities benign or simply ignore/truncate the quantities of all clients. We propose a robust quantity-aware aggregation algorithm for federated learning, called FedRA, to perform the aggregation with awareness of local data quantities.
  arXiv  Detail & Related papers  (2022-05-22T15:13:23Z)
• SwiftAgg: Communication-Efficient and Dropout-Resistant Secure Aggregation for Federated Learning with Worst-Case Security Guarantees [83.94234859890402]
  We propose SwiftAgg, a novel secure aggregation protocol for federated learning systems. A central server aggregates local models of $N$ distributed users, each of size $L$, trained on their local data. SwiftAgg significantly reduces the communication overheads without any compromise on security.
  arXiv  Detail & Related papers  (2022-02-08T22:08:56Z)
• LightSecAgg: Rethinking Secure Aggregation in Federated Learning [24.834891926133594]
  We show that LightSecAgg achieves the same privacy and dropout-resiliency guarantees as the state-of-the-art protocols. We also demonstrate that LightSecAgg significantly reduces the total training time, achieving a performance gain of up to $12.7times$ over baselines.
  arXiv  Detail & Related papers  (2021-09-29T07:19:27Z)
• Privacy Amplification for Federated Learning via User Sampling and Wireless Aggregation [17.56067859013419]
  We study the problem of federated learning over a wireless channel with user sampling. We propose a private wireless gradient aggregation scheme, which relies on independently randomized participation decisions by each user.
  arXiv  Detail & Related papers  (2021-03-02T18:59:37Z)
• FastSecAgg: Scalable Secure Aggregation for Privacy-Preserving Federated Learning [18.237186837994585]
  A'secure aggregation' protocol enables the server to aggregate clients' models in a privacy-preserving manner. FastSecAgg is efficient in terms of computation and communication, and robust to client dropouts.
  arXiv  Detail & Related papers  (2020-09-23T16:49:02Z)
• Multi-Center Federated Learning [62.57229809407692]
  This paper proposes a novel multi-center aggregation mechanism for federated learning. It learns multiple global models from the non-IID user data and simultaneously derives the optimal matching between users and centers. Our experimental results on benchmark datasets show that our method outperforms several popular federated learning methods.
  arXiv  Detail & Related papers  (2020-05-03T09:14:31Z)
• Robust Aggregation for Federated Learning [37.47208810846432]
  Federated learning is the centralized training of statistical models from decentralized data on mobile devices. We present a robust aggregation approach to make federated learning robust to settings when a fraction of the devices may be sending corrupted updates to the server.
  arXiv  Detail & Related papers  (2019-12-31T17:24:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.