Enhancing Privacy in Federated Learning through Local Training

• URL: http://arxiv.org/abs/2403.17572v2
• Date: Thu, 28 Nov 2024 08:19:51 GMT
• Title: Enhancing Privacy in Federated Learning through Local Training
• Abstract summary: We propose the federated learning algorithm Fed-PLT to overcome the challenges of (i) expensive communications and (ii) privacy preservation.<n>We address (i) by allowing for both partial participation and local training, which significantly reduce the number of communication rounds between the central coordinator and computing agents.
• Score: 2.1096737598952853
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper we propose the federated learning algorithm Fed-PLT to overcome the challenges of (i) expensive communications and (ii) privacy preservation. We address (i) by allowing for both partial participation and local training, which significantly reduce the number of communication rounds between the central coordinator and computing agents. The algorithm matches the state of the art in the sense that the use of local training demonstrably does not impact accuracy. Additionally, agents have the flexibility to choose from various local training solvers, such as (stochastic) gradient descent and accelerated gradient descent. Further, we investigate how employing local training can enhance privacy, addressing point (ii). In particular, we derive differential privacy bounds and highlight their dependence on the number of local training epochs. We assess the effectiveness of the proposed algorithm by comparing it to alternative techniques, considering both theoretical analysis and numerical results from a classification task.

Related papers

• DPDL: Towards Differential Privacy Preservation in Decentralized Stochastic Learning on Non-IID Data [5.903492793014408]
  In the paradigm of decentralized learning, a group of agents collaborate to train a global model using distributed datasets without a central server.<n>We propose a privacy-preserved decentralized learning algorithm with non-IID data, DPDL, which leverages the notion of Differential Privacy (DP) in cross-gradient aggregation.<n>Our rigorous theoretical analysis not only reveals the minimum noise level required to achieve a specific level of privacy preservation, but also illustrates that our algorithm still achieves a linear speedup in training with non-IID data.
  arXiv  Detail & Related papers  (2026-06-03T03:27:40Z)
• Communication-Efficient Distributed Learning with Differential Privacy [4.520252012039036]
  We focus on the challenge of designing an algorithm that is both communication-efficient and that guarantees the privacy of the agents' data.<n>We prove that the algorithm converges to a stationary point of the problem within a bounded distance.<n>We show the algorithm's superior performance on a classification task under the same privacy budget, compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-04-02T22:19:58Z)
• Jointly Computation- and Communication-Efficient Distributed Learning [3.2923780772605595]
  We focus on designing a novel ADMM-based algorithm that is jointly computation and communication-efficient.<n>Our design guarantees computational efficiency by allowing agents to use gradients during local training.<n>We prove exact linear convergence of the algorithm in the strongly convex setting.
  arXiv  Detail & Related papers  (2025-08-21T12:36:02Z)
• Learning Multi-Robot Coordination through Locality-Based Factorized Multi-Agent Actor-Critic Algorithm [54.98788921815576]
  We present a novel cooperative multi-agent reinforcement learning method called textbfLocality based textbfFactorized textbfMulti-Agent textbfActor-textbfCritic (Loc-FACMAC)<n>We integrate the concept of locality into critic learning, where strongly related robots form partitions during training.<n>Our method improves existing algorithms by focusing on local rewards and leveraging partition-based learning to enhance training efficiency and performance.
  arXiv  Detail & Related papers  (2025-03-24T16:00:16Z)
• Locally Differentially Private Gradient Tracking for Distributed Online Learning over Directed Graphs [2.1271873498506038]
  We propose a locally differentially private gradient tracking based distributed online learning algorithm. We prove that the proposed algorithm converges in mean square to the exact optimal solution while ensuring rigorous local differential privacy.
  arXiv  Detail & Related papers  (2023-10-24T18:15:25Z)
• FedLALR: Client-Specific Adaptive Learning Rates Achieve Linear Speedup for Non-IID Data [54.81695390763957]
  Federated learning is an emerging distributed machine learning method. We propose a heterogeneous local variant of AMSGrad, named FedLALR, in which each client adjusts its learning rate. We show that our client-specified auto-tuned learning rate scheduling can converge and achieve linear speedup with respect to the number of clients.
  arXiv  Detail & Related papers  (2023-09-18T12:35:05Z)
• Online Distributed Learning with Quantized Finite-Time Coordination [0.4910937238451484]
  In our setting a set of agents need to cooperatively train a learning model from streaming data. We propose a distributed algorithm that relies on a quantized, finite-time coordination protocol. We analyze the performance of the proposed algorithm in terms of the mean distance from the online solution.
  arXiv  Detail & Related papers  (2023-07-13T08:36:15Z)
• Explicit Personalization and Local Training: Double Communication Acceleration in Federated Learning [7.691755449724637]
  A common strategy to curtail communication costs is Local Training, which consists in performing multiple local gradient descent steps between successive communication rounds. We introduce Scafflix, a novel algorithm that efficiently integrates explicit personalization with local training.
  arXiv  Detail & Related papers  (2023-05-22T15:58:01Z)
• DPP-based Client Selection for Federated Learning with Non-IID Data [97.1195165400568]
  This paper proposes a client selection (CS) method to tackle the communication bottleneck of federated learning (FL) We first analyze the effect of CS in FL and show that FL training can be accelerated by adequately choosing participants to diversify the training dataset in each round of training. We leverage data profiling and determinantal point process (DPP) sampling techniques to develop an algorithm termed Federated Learning with DPP-based Participant Selection (FL-DP$3$S)
  arXiv  Detail & Related papers  (2023-03-30T13:14:54Z)
• Exploring Low Rank Training of Deep Neural Networks [49.18122605463354]
  Training deep neural networks in low rank offers efficiency over unfactorised training in terms of both memory consumption and training time. We analyse techniques that work well in practice, and through extensive ablations on models such as GPT2 we provide evidence falsifying common beliefs in the field.
  arXiv  Detail & Related papers  (2022-09-27T17:43:45Z)
• Preserving Privacy in Federated Learning with Ensemble Cross-Domain Knowledge Distillation [22.151404603413752]
  Federated Learning (FL) is a machine learning paradigm where local nodes collaboratively train a central model. Existing FL methods typically share model parameters or employ co-distillation to address the issue of unbalanced data distribution. We develop a privacy preserving and communication efficient method in a FL framework with one-shot offline knowledge distillation.
  arXiv  Detail & Related papers  (2022-09-10T05:20:31Z)
• Certified Robustness in Federated Learning [54.03574895808258]
  We study the interplay between federated training, personalization, and certified robustness. We find that the simple federated averaging technique is effective in building not only more accurate, but also more certifiably-robust models.
  arXiv  Detail & Related papers  (2022-06-06T12:10:53Z)
• DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training [84.81043932706375]
  We propose a novel personalized federated learning framework in a decentralized (peer-to-peer) communication protocol named Dis-PFL. Dis-PFL employs personalized sparse masks to customize sparse local models on the edge. We demonstrate that our method can easily adapt to heterogeneous local clients with varying computation complexities.
  arXiv  Detail & Related papers  (2022-06-01T02:20:57Z)
• Acceleration of Federated Learning with Alleviated Forgetting in Local Training [61.231021417674235]
  Federated learning (FL) enables distributed optimization of machine learning models while protecting privacy. We propose FedReg, an algorithm to accelerate FL with alleviated knowledge forgetting in the local training stage. Our experiments demonstrate that FedReg not only significantly improves the convergence rate of FL, especially when the neural network architecture is deep.
  arXiv  Detail & Related papers  (2022-03-05T02:31:32Z)
• Weight Divergence Driven Divide-and-Conquer Approach for Optimal Federated Learning from non-IID Data [0.0]
  Federated Learning allows training of data stored in distributed devices without the need for centralizing training data. We propose a novel Divide-and-Conquer training methodology that enables the use of the popular FedAvg aggregation algorithm.
  arXiv  Detail & Related papers  (2021-06-28T09:34:20Z)
• Learning Calibrated Uncertainties for Domain Shift: A Distributionally Robust Learning Approach [150.8920602230832]
  We propose a framework for learning calibrated uncertainties under domain shifts. In particular, the density ratio estimation reflects the closeness of a target (test) sample to the source (training) distribution. We show that our proposed method generates calibrated uncertainties that benefit downstream tasks.
  arXiv  Detail & Related papers  (2020-10-08T02:10:54Z)
• Adaptive Serverless Learning [114.36410688552579]
  We propose a novel adaptive decentralized training approach, which can compute the learning rate from data dynamically. Our theoretical results reveal that the proposed algorithm can achieve linear speedup with respect to the number of workers. To reduce the communication-efficient overhead, we further propose a communication-efficient adaptive decentralized training approach.
  arXiv  Detail & Related papers  (2020-08-24T13:23:02Z)
• Federated Learning for Localization: A Privacy-Preserving Crowdsourcing Method [0.0]
  Received Signal Strength ( RSS) fingerprint-based localization has attracted a lot of research effort and cultivated many commercial applications. DL's ability to extract features and to classify autonomously makes it an attractive solution for fingerprint-based localization. This paper presents a novel method utilizing federated learning to improve the accuracy of RSS fingerprint-based localization.
  arXiv  Detail & Related papers  (2020-01-07T07:02:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.