Communication-Efficient Distributed Learning with Differential Privacy

• URL: http://arxiv.org/abs/2604.02558v1
• Date: Thu, 02 Apr 2026 22:19:58 GMT
• Title: Communication-Efficient Distributed Learning with Differential Privacy
• Abstract summary: We focus on the challenge of designing an algorithm that is both communication-efficient and that guarantees the privacy of the agents' data.<n>We prove that the algorithm converges to a stationary point of the problem within a bounded distance.<n>We show the algorithm's superior performance on a classification task under the same privacy budget, compared with state-of-the-art methods.
• Score: 4.520252012039036
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We address nonconvex learning problems over undirected networks. In particular, we focus on the challenge of designing an algorithm that is both communication-efficient and that guarantees the privacy of the agents' data. The first goal is achieved through a local training approach, which reduces communication frequency. The second goal is achieved by perturbing gradients during local training, specifically through gradient clipping and additive noise. We prove that the resulting algorithm converges to a stationary point of the problem within a bounded distance. Additionally, we provide theoretical privacy guarantees within a differential privacy framework that ensure agents' training data cannot be inferred from the trained model shared over the network. We show the algorithm's superior performance on a classification task under the same privacy budget, compared with state-of-the-art methods.

Related papers

• Federated Learning with Differential Privacy: An Utility-Enhanced Approach [12.614480013684759]
  Federated learning has emerged as an attractive approach to protect data privacy by eliminating the need for sharing clients' data.<n>Recent studies have shown that federated learning alone does not guarantee privacy, as private data may still be inferred from the uploaded parameters to the central server.<n>We present a modification to these vanilla differentially private algorithms based on a Haar wavelet transformation step and a novel noise injection scheme that significantly lowers the bound of the noise variance.
  arXiv  Detail & Related papers  (2025-03-27T04:48:29Z)
• Collaborative Inference over Wireless Channels with Feature Differential Privacy [57.68286389879283]
  Collaborative inference among multiple wireless edge devices has the potential to significantly enhance Artificial Intelligence (AI) applications. transmitting extracted features poses a significant privacy risk, as sensitive personal data can be exposed during the process. We propose a novel privacy-preserving collaborative inference mechanism, wherein each edge device in the network secures the privacy of extracted features before transmitting them to a central server for inference.
  arXiv  Detail & Related papers  (2024-10-25T18:11:02Z)
• Enhancing Privacy in Federated Learning through Local Training [2.1096737598952853]
  We propose the federated learning algorithm Fed-PLT to overcome the challenges of (i) expensive communications and (ii) privacy preservation.<n>We address (i) by allowing for both partial participation and local training, which significantly reduce the number of communication rounds between the central coordinator and computing agents.
  arXiv  Detail & Related papers  (2024-03-26T10:25:21Z)
• Locally Differentially Private Gradient Tracking for Distributed Online Learning over Directed Graphs [2.1271873498506038]
  We propose a locally differentially private gradient tracking based distributed online learning algorithm. We prove that the proposed algorithm converges in mean square to the exact optimal solution while ensuring rigorous local differential privacy.
  arXiv  Detail & Related papers  (2023-10-24T18:15:25Z)
• Sparse Decentralized Federated Learning [35.32297764027417]
  Decentralized Federated Learning (DFL) enables collaborative model training without a central server but faces challenges in efficiency, stability, and trustworthiness.<n>We introduce a sparsity constraint on the shared model, leading to Sparse DFL (SDFL), and propose a novel algorithm, CEPS.<n> Numerical experiments validate the effectiveness of the proposed algorithm in improving communication and efficiency while maintaining a high level of trustworthiness.
  arXiv  Detail & Related papers  (2023-08-31T12:22:40Z)
• Locally Differentially Private Distributed Online Learning with Guaranteed Optimality [1.800614371653704]
  This paper proposes an approach that ensures both differential privacy and learning accuracy in distributed online learning. While ensuring a diminishing expected instantaneous regret, the approach can simultaneously ensure a finite cumulative privacy budget. To the best of our knowledge, this is the first algorithm that successfully ensures both rigorous local differential privacy and learning accuracy.
  arXiv  Detail & Related papers  (2023-06-25T02:05:34Z)
• Differentially Private Stochastic Gradient Descent with Low-Noise [49.981789906200035]
  Modern machine learning algorithms aim to extract fine-grained information from data to provide accurate predictions, which often conflicts with the goal of privacy protection. This paper addresses the practical and theoretical importance of developing privacy-preserving machine learning algorithms that ensure good performance while preserving privacy.
  arXiv  Detail & Related papers  (2022-09-09T08:54:13Z)
• sqSGD: Locally Private and Communication Efficient Federated Learning [14.60645909629309]
  Federated learning (FL) is a technique that trains machine learning models from decentralized data sources. We develop a gradient-based learning algorithm called sqSGD that addresses communication efficiency and high-dimensional compatibility. Experiment results show sqSGD successfully learns large models like LeNet and ResNet with local privacy constraints.
  arXiv  Detail & Related papers  (2022-06-21T17:45:35Z)
• Federated Learning for Face Recognition with Gradient Correction [52.896286647898386]
  In this work, we introduce a framework, FedGC, to tackle federated learning for face recognition. We show that FedGC constitutes a valid loss function similar to standard softmax.
  arXiv  Detail & Related papers  (2021-12-14T09:19:29Z)
• Unsupervised Domain-adaptive Hash for Networks [81.49184987430333]
  Domain-adaptive hash learning has enjoyed considerable success in the computer vision community. We develop an unsupervised domain-adaptive hash learning method for networks, dubbed UDAH.
  arXiv  Detail & Related papers  (2021-08-20T12:09:38Z)
• TraND: Transferable Neighborhood Discovery for Unsupervised Cross-domain Gait Recognition [77.77786072373942]
  This paper proposes a Transferable Neighborhood Discovery (TraND) framework to bridge the domain gap for unsupervised cross-domain gait recognition. We design an end-to-end trainable approach to automatically discover the confident neighborhoods of unlabeled samples in the latent space. Our method achieves state-of-the-art results on two public datasets, i.e., CASIA-B and OU-LP.
  arXiv  Detail & Related papers  (2021-02-09T03:07:07Z)
• Graph-Homomorphic Perturbations for Private Decentralized Learning [64.26238893241322]
  Local exchange of estimates allows inference of data based on private data. perturbations chosen independently at every agent, resulting in a significant performance loss. We propose an alternative scheme, which constructs perturbations according to a particular nullspace condition, allowing them to be invisible.
  arXiv  Detail & Related papers  (2020-10-23T10:35:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.