Multi-Objective Optimization for Privacy-Utility Balance in Differentially Private Federated Learning

• URL: http://arxiv.org/abs/2503.21159v1
• Date: Thu, 27 Mar 2025 04:57:05 GMT
• Title: Multi-Objective Optimization for Privacy-Utility Balance in Differentially Private Federated Learning
• Authors: Kanishka Ranaweera, David Smith, Pubudu N. Pathirana, Ming Ding, Thierry Rakotoarivelo, Aruna Seneviratne,
• Abstract summary: Federated learning (FL) enables collaborative model training across distributed clients without sharing raw data.<n>We propose an adaptive clipping mechanism that dynamically adjusts the clipping norm using a multi-objective optimization framework.<n>Our results show that adaptive clipping consistently outperforms fixed-clipping baselines, achieving improved accuracy under the same privacy constraints.
• Score: 12.278668095136098
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated learning (FL) enables collaborative model training across distributed clients without sharing raw data, making it a promising approach for privacy-preserving machine learning. However, ensuring differential privacy (DP) in FL presents challenges due to the trade-off between model utility and privacy protection. Clipping gradients before aggregation is a common strategy to limit privacy loss, but selecting an optimal clipping norm is non-trivial, as excessively high values compromise privacy, while overly restrictive clipping degrades model performance. In this work, we propose an adaptive clipping mechanism that dynamically adjusts the clipping norm using a multi-objective optimization framework. By integrating privacy and utility considerations into the optimization objective, our approach balances privacy preservation with model accuracy. We theoretically analyze the convergence properties of our method and demonstrate its effectiveness through extensive experiments on MNIST, Fashion-MNIST, and CIFAR-10 datasets. Our results show that adaptive clipping consistently outperforms fixed-clipping baselines, achieving improved accuracy under the same privacy constraints. This work highlights the potential of dynamic clipping strategies to enhance privacy-utility trade-offs in differentially private federated learning.

Related papers

• Adaptive Clipping for Privacy-Preserving Few-Shot Learning: Enhancing Generalization with Limited Data [12.614480013684759]
  We introduce a novel approach called Meta-Clip to enhance the utility of privacy-preserving few-shot learning methods. By dynamically adjusting clipping thresholds during the training process, our Adaptive Clipping method provides fine-grained control over the disclosure of sensitive information. We demonstrate the effectiveness of our approach in minimizing utility degradation, showcasing a superior privacy-preserving trade-off compared to existing privacy-preserving techniques.
  arXiv  Detail & Related papers  (2025-03-27T05:14:18Z)
• Personalized Language Models via Privacy-Preserving Evolutionary Model Merging [57.161917758405465]
  Personalization in large language models (LLMs) seeks to tailor models to individual user or user group preferences.<n>We propose Privacy-Preserving Model Merging via Evolutionary Algorithms (PriME)<n>PriME employs gradient-free methods to directly optimize task-specific metrics while preserving user privacy.
  arXiv  Detail & Related papers  (2025-03-23T09:46:07Z)
• FedAWA: Adaptive Optimization of Aggregation Weights in Federated Learning Using Client Vectors [50.131271229165165]
  Federated Learning (FL) has emerged as a promising framework for distributed machine learning. Data heterogeneity resulting from differences across user behaviors, preferences, and device characteristics poses a significant challenge for federated learning. We propose Adaptive Weight Aggregation (FedAWA), a novel method that adaptively adjusts aggregation weights based on client vectors during the learning process.
  arXiv  Detail & Related papers  (2025-03-20T04:49:40Z)
• Differential Privacy Personalized Federated Learning Based on Dynamically Sparsified Client Updates [12.373620724244475]
  We propose a differentially private personalized federated learning approach that employs dynamically sparsified client updates.<n> Experimental results on EMNIST, CIFAR-10, and CIFAR-100 demonstrate that our proposed scheme achieves superior performance.
  arXiv  Detail & Related papers  (2025-03-12T09:34:05Z)
• Advancing Personalized Federated Learning: Integrative Approaches with AI for Enhanced Privacy and Customization [0.0]
  This paper proposes a novel approach that enhances PFL with cutting-edge AI techniques.<n>We present a model that boosts the performance of individual client models and ensures robust privacy-preserving mechanisms.<n>This work paves the way for a new era of truly personalized and privacy-conscious AI systems.
  arXiv  Detail & Related papers  (2025-01-30T07:03:29Z)
• Few-shot Steerable Alignment: Adapting Rewards and LLM Policies with Neural Processes [50.544186914115045]
  Large language models (LLMs) are increasingly embedded in everyday applications. Ensuring their alignment with the diverse preferences of individual users has become a critical challenge. We present a novel framework for few-shot steerable alignment.
  arXiv  Detail & Related papers  (2024-12-18T16:14:59Z)
• CorBin-FL: A Differentially Private Federated Learning Mechanism using Common Randomness [6.881974834597426]
  Federated learning (FL) has emerged as a promising framework for distributed machine learning. We introduce CorBin-FL, a privacy mechanism that uses correlated binary quantization to achieve differential privacy. We also propose AugCorBin-FL, an extension that, in addition to PLDP, user-level and sample-level central differential privacy guarantees.
  arXiv  Detail & Related papers  (2024-09-20T00:23:44Z)
• Differentially Private Deep Model-Based Reinforcement Learning [47.651861502104715]
  We introduce PriMORL, a model-based RL algorithm with formal differential privacy guarantees. PriMORL learns an ensemble of trajectory-level DP models of the environment from offline data.
  arXiv  Detail & Related papers  (2024-02-08T10:05:11Z)
• Online Sensitivity Optimization in Differentially Private Learning [8.12606646175019]
  We present a novel approach to dynamically optimize the clipping threshold. We treat this threshold as an additional learnable parameter, establishing a clean relationship between the threshold and the cost function. Our method is thoroughly assessed against alternative fixed and adaptive strategies across diverse datasets, tasks, model dimensions, and privacy levels.
  arXiv  Detail & Related papers  (2023-10-02T00:30:49Z)
• Theoretically Principled Federated Learning for Balancing Privacy and Utility [61.03993520243198]
  We propose a general learning framework for the protection mechanisms that protects privacy via distorting model parameters. It can achieve personalized utility-privacy trade-off for each model parameter, on each client, at each communication round in federated learning.
  arXiv  Detail & Related papers  (2023-05-24T13:44:02Z)
• Personalized Federated Learning under Mixture of Distributions [98.25444470990107]
  We propose a novel approach to Personalized Federated Learning (PFL), which utilizes Gaussian mixture models (GMM) to fit the input data distributions across diverse clients. FedGMM possesses an additional advantage of adapting to new clients with minimal overhead, and it also enables uncertainty quantification. Empirical evaluations on synthetic and benchmark datasets demonstrate the superior performance of our method in both PFL classification and novel sample detection.
  arXiv  Detail & Related papers  (2023-05-01T20:04:46Z)
• FedLAP-DP: Federated Learning by Sharing Differentially Private Loss Approximations [53.268801169075836]
  We propose FedLAP-DP, a novel privacy-preserving approach for federated learning. A formal privacy analysis demonstrates that FedLAP-DP incurs the same privacy costs as typical gradient-sharing schemes. Our approach presents a faster convergence speed compared to typical gradient-sharing methods.
  arXiv  Detail & Related papers  (2023-02-02T12:56:46Z)
• On the utility and protection of optimization with differential privacy and classic regularization techniques [9.413131350284083]
  We study the effectiveness of the differentially-private descent (DP-SGD) algorithm against standard optimization practices with regularization techniques. We discuss differential privacy's flaws and limits and empirically demonstrate the often superior privacy-preserving properties of dropout and l2-regularization.
  arXiv  Detail & Related papers  (2022-09-07T14:10:21Z)
• Differentially Private Federated Learning with Laplacian Smoothing [72.85272874099644]
  Federated learning aims to protect data privacy by collaboratively learning a model without sharing private data among users. An adversary may still be able to infer the private training data by attacking the released model. Differential privacy provides a statistical protection against such attacks at the price of significantly degrading the accuracy or utility of the trained models.
  arXiv  Detail & Related papers  (2020-05-01T04:28:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.