Communication-Learning Co-Design for Differentially Private Over-the-Air Federated Distillation

• URL: http://arxiv.org/abs/2508.06557v1
• Date: Wed, 06 Aug 2025 12:44:10 GMT
• Title: Communication-Learning Co-Design for Differentially Private Over-the-Air Federated Distillation
• Authors: Zihao Hu, Jia Yan, Ying-Jun Angela Zhang,
• Abstract summary: We study the communication-learning co-design problem in differentially private over-the-air FD, aiming to maximize the learning convergence rate.<n> Numerical results demonstrate that the proposed differentially private over-the-air FD approach achieves a better learning-privacy trade-off with largely-reduced communication overhead.
• Score: 23.455662261351193
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The ever-growing learning model size nowadays challenges the communication efficiency and privacy preservation of the traditional federated learning (FL). In this paper, we propose a novel differentially private (DP) over-the-air federated distillation (FD) framework, where wireless devices (WDs) periodically share noise-perturbed model outputs with the parameter server by harnessing the superposition property of multi-access channels. Accordingly, over-the-air FD enables the shared responsibility of the DP preservation on the low-dimensional disclosed signals among WDs. We study the communication-learning co-design problem in differentially private over-the-air FD, aiming to maximize the learning convergence rate while meeting the transmit power and DP requirements of WDs. The main challenge is rooted in the intractable learning and privacy analysis in over-the-air FD, together with the strong coupling among the decision variables spanning two timescales. To tackle this problem, we first derive the analytical learning convergence rate and privacy losses of WDs, based on which the optimal transceiver design per FD round and long-term training rounds decision are obtained in the closed forms. Numerical results demonstrate that the proposed differentially private over-the-air FD approach achieves a better learning-privacy trade-off with largely-reduced communication overhead than the conventional FL benchmarks.

Related papers

• Optimal Transceiver Design in Over-the-Air Federated Distillation [34.09979141255862]
  We study the transceiver design in terms of the learning convergence rate and the power constraints.<n>We propose a novel approach to find the optimal receiver beam vector for over-the-air aggregation.<n>Results show that the proposed over-the-air approach achieves a significant reduction in communication with only a minor compromise in testing accuracy.
  arXiv  Detail & Related papers  (2025-07-21T05:37:08Z)
• Embedding-Based Federated Data Sharing via Differentially Private Conditional VAEs [0.13108652488669734]
  Federated Learning (FL) enables decentralized training but suffers from high communication costs.<n>We propose a data-sharing method via Differentially Private (DP) generative models.<n>Clients collaboratively train a Differentially Private Conditional Variational Autoencoder (DP-CVAE) to model a global, privacy-aware data distribution.
  arXiv  Detail & Related papers  (2025-07-03T14:36:15Z)
• Convergent Differential Privacy Analysis for General Federated Learning: the $f$-DP Perspective [57.35402286842029]
  Federated learning (FL) is an efficient collaborative training paradigm with a focus on local privacy. differential privacy (DP) is a classical approach to capture and ensure the reliability of private protections.
  arXiv  Detail & Related papers  (2024-08-28T08:22:21Z)
• Universally Harmonizing Differential Privacy Mechanisms for Federated Learning: Boosting Accuracy and Convergence [22.946928984205588]
  Differentially private federated learning (DP-FL) is a promising technique for collaborative model training. We propose the first DP-FL framework (namely UDP-FL) which universally harmonizes any randomization mechanism. We show that UDP-FL exhibits substantial resilience against different inference attacks.
  arXiv  Detail & Related papers  (2024-07-20T00:11:59Z)
• Personalized Wireless Federated Learning for Large Language Models [75.22457544349668]
  Large language models (LLMs) have driven profound transformations in wireless networks.<n>Within wireless environments, the training of LLMs faces significant challenges related to security and privacy.<n>This paper presents a systematic analysis of the training stages of LLMs in wireless networks, including pre-training, instruction tuning, and alignment tuning.
  arXiv  Detail & Related papers  (2024-04-20T02:30:21Z)
• Federated Distillation: A Survey [73.08661634882195]
  Federated Learning (FL) seeks to train a model collaboratively without sharing private training data from individual clients. The integration of knowledge distillation into FL has been proposed, forming what is known as Federated Distillation (FD) FD enables more flexible knowledge transfer between clients and the server, surpassing the mere sharing of model parameters.
  arXiv  Detail & Related papers  (2024-04-02T03:42:18Z)
• Differentially Private Over-the-Air Federated Learning Over MIMO Fading Channels [24.534729104570417]
  Federated learning (FL) enables edge devices to collaboratively train machine learning models. While over-the-air model aggregation improves communication efficiency, uploading models to an edge server over wireless networks can pose privacy risks. We show that FL model communication with a multiple-antenna server amplifies privacy leakage.
  arXiv  Detail & Related papers  (2023-06-19T14:44:34Z)
• Differentially Private Wireless Federated Learning Using Orthogonal Sequences [56.52483669820023]
  We propose a privacy-preserving uplink over-the-air computation (AirComp) method, termed FLORAS. We prove that FLORAS offers both item-level and client-level differential privacy guarantees. A new FL convergence bound is derived which, combined with the privacy guarantees, allows for a smooth tradeoff between the achieved convergence rate and differential privacy levels.
  arXiv  Detail & Related papers  (2023-06-14T06:35:10Z)
• Low-Latency Federated Learning over Wireless Channels with Differential Privacy [142.5983499872664]
  In federated learning (FL), model training is distributed over clients and local models are aggregated by a central server. In this paper, we aim to minimize FL training delay over wireless channels, constrained by overall training performance as well as each client's differential privacy (DP) requirement.
  arXiv  Detail & Related papers  (2021-06-20T13:51:18Z)
• Voting-based Approaches For Differentially Private Federated Learning [87.2255217230752]
  This work is inspired by knowledge transfer non-federated privacy learning from Papernot et al. We design two new DPFL schemes, by voting among the data labels returned from each local model, instead of averaging the gradients. Our approaches significantly improve the privacy-utility trade-off over the state-of-the-arts in DPFL.
  arXiv  Detail & Related papers  (2020-10-09T23:55:19Z)
• Differentially Private Federated Learning with Laplacian Smoothing [72.85272874099644]
  Federated learning aims to protect data privacy by collaboratively learning a model without sharing private data among users. An adversary may still be able to infer the private training data by attacking the released model. Differential privacy provides a statistical protection against such attacks at the price of significantly degrading the accuracy or utility of the trained models.
  arXiv  Detail & Related papers  (2020-05-01T04:28:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.