Learning with Differentially Private (Sliced) Wasserstein Gradients

• URL: http://arxiv.org/abs/2502.01701v1
• Date: Mon, 03 Feb 2025 09:14:26 GMT
• Title: Learning with Differentially Private (Sliced) Wasserstein Gradients
• Authors: Clément Lalanne, Jean-Michel Loubes, David Rodríguez-Vítores,
• Abstract summary: We introduce a novel framework for privately optimizing objectives that rely on Wasserstein distances between data-dependent empirical measures. Our main theoretical contribution is, based on an explicit formulation of the Wasserstein gradient in a fully discrete setting. We develop a deep learning approach that incorporates gradient and activations clipping, originally designed for DP training of problems with a finite-sum structure.
• Score: 3.154269505086155
• License:
• Abstract: In this work, we introduce a novel framework for privately optimizing objectives that rely on Wasserstein distances between data-dependent empirical measures. Our main theoretical contribution is, based on an explicit formulation of the Wasserstein gradient in a fully discrete setting, a control on the sensitivity of this gradient to individual data points, allowing strong privacy guarantees at minimal utility cost. Building on these insights, we develop a deep learning approach that incorporates gradient and activations clipping, originally designed for DP training of problems with a finite-sum structure. We further demonstrate that privacy accounting methods extend to Wasserstein-based objectives, facilitating large-scale private training. Empirical results confirm that our framework effectively balances accuracy and privacy, offering a theoretically sound solution for privacy-preserving machine learning tasks relying on optimal transport distances such as Wasserstein distance or sliced-Wasserstein distance.

Related papers

• Linear-Time User-Level DP-SCO via Robust Statistics [55.350093142673316]
  User-level differentially private convex optimization (DP-SCO) has garnered significant attention due to the importance of safeguarding user privacy in machine learning applications. Current methods, such as those based on differentially private gradient descent (DP-SGD), often struggle with high noise accumulation and suboptimal utility. We introduce a novel linear-time algorithm that leverages robust statistics, specifically the median and trimmed mean, to overcome these challenges.
  arXiv  Detail & Related papers  (2025-02-13T02:05:45Z)
• Distributed, communication-efficient, and differentially private estimation of KL divergence [15.294136011320433]
  Key task in managing distributed, sensitive data is to measure the extent to which a distribution changes. We describe novel algorithmic approaches for estimating the KL divergence of data across federated models of computation, under differential privacy.
  arXiv  Detail & Related papers  (2024-11-25T15:20:40Z)
• Private Wasserstein Distance [6.015898117103069]
  Wasserstein distance is a key metric for quantifying data divergence from a distributional perspective. In this study, we explore the inherent triangular properties within the Wasserstein space, leading to a novel solution named TriangleWad.
  arXiv  Detail & Related papers  (2024-04-10T06:58:58Z)
• Differentially Private Gradient Flow based on the Sliced Wasserstein Distance [59.1056830438845]
  We introduce a novel differentially private generative modeling approach based on a gradient flow in the space of probability measures. Experiments show that our proposed model can generate higher-fidelity data at a low privacy budget.
  arXiv  Detail & Related papers  (2023-12-13T15:47:30Z)
• Theoretically Principled Federated Learning for Balancing Privacy and Utility [61.03993520243198]
  We propose a general learning framework for the protection mechanisms that protects privacy via distorting model parameters. It can achieve personalized utility-privacy trade-off for each model parameter, on each client, at each communication round in federated learning.
  arXiv  Detail & Related papers  (2023-05-24T13:44:02Z)
• On Mitigating the Utility-Loss in Differentially Private Learning: A new Perspective by a Geometrically Inspired Kernel Approach [2.4253452809863116]
  This paper introduces a geometrically inspired kernel-based approach to mitigate the accuracy-loss issue in classification. A representation of the affine hull of given data points is learned in Reproducing Kernel Hilbert Spaces (RKHS) The effectiveness of the approach is demonstrated through experiments on MNIST dataset, Freiburg groceries dataset, and a real biomedical dataset.
  arXiv  Detail & Related papers  (2023-04-03T18:52:01Z)
• An automatic differentiation system for the age of differential privacy [65.35244647521989]
  Tritium is an automatic differentiation-based sensitivity analysis framework for differentially private (DP) machine learning (ML) We introduce Tritium, an automatic differentiation-based sensitivity analysis framework for differentially private (DP) machine learning (ML)
  arXiv  Detail & Related papers  (2021-09-22T08:07:42Z)
• Differentially Private Sliced Wasserstein Distance [5.330240017302619]
  We take the perspective of computing the divergences between distributions under the Differential Privacy (DP) framework. Instead of resorting to the popular gradient-based sanitization method for DP, we tackle the problem at its roots by focusing on the Sliced Wasserstein Distance.
  arXiv  Detail & Related papers  (2021-07-05T08:06:02Z)
• Do Not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learning [74.73901662374921]
  A differentially private model degrades the utility drastically when the model comprises a large number of trainable parameters. We propose an algorithm emphGradient Embedding Perturbation (GEP) towards training differentially private deep models with decent accuracy.
  arXiv  Detail & Related papers  (2021-02-25T04:29:58Z)
• Privacy-Constrained Policies via Mutual Information Regularized Policy Gradients [54.98496284653234]
  We consider the task of training a policy that maximizes reward while minimizing disclosure of certain sensitive state variables through the actions. We solve this problem by introducing a regularizer based on the mutual information between the sensitive state and the actions. We develop a model-based estimator for optimization of privacy-constrained policies.
  arXiv  Detail & Related papers  (2020-12-30T03:22:35Z)
• SPEED: Secure, PrivatE, and Efficient Deep learning [2.283665431721732]
  We introduce a deep learning framework able to deal with strong privacy constraints. Based on collaborative learning, differential privacy and homomorphic encryption, the proposed approach advances state-of-the-art.
  arXiv  Detail & Related papers  (2020-06-16T19:31:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.