Related papers: Adversarial Sample-Based Approach for Tighter Privacy Auditing in Final Model-Only Scenarios

Adversarial Sample-Based Approach for Tighter Privacy Auditing in Final Model-Only Scenarios

URL: http://arxiv.org/abs/2412.01756v2
Date: Mon, 24 Feb 2025 07:27:11 GMT
Title: Adversarial Sample-Based Approach for Tighter Privacy Auditing in Final Model-Only Scenarios
Authors: Sangyeon Yoon, Wonje Jeung, Albert No,
Abstract summary: We introduce a novel auditing method that achieves tighter empirical lower bounds without additional assumptions.<n>Our approach surpasses traditional canary-based adversarials and is effective in final model-only scenarios.
Score: 5.116399056871577
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Auditing Differentially Private Stochastic Gradient Descent (DP-SGD) in the final model setting is challenging and often results in empirical lower bounds that are significantly looser than theoretical privacy guarantees. We introduce a novel auditing method that achieves tighter empirical lower bounds without additional assumptions by crafting worst-case adversarial samples through loss-based input-space auditing. Our approach surpasses traditional canary-based heuristics and is effective in final model-only scenarios. Specifically, with a theoretical privacy budget of $\varepsilon = 10.0$, our method achieves empirical lower bounds of $4.914$, compared to the baseline of $4.385$ for MNIST. Our work offers a practical framework for reliable and accurate privacy auditing in differentially private machine learning.

Related papers

Sequential Auditing for f-Differential Privacy [5.7992233755396505]
We present new auditors to assess Differential Privacy (DP) of an algorithm based on output samples.<n>We shift the focus to the highly expressive privacy concept of $f$-DP, in which the entire privacy behavior is captured by a single tradeoff curve.
arXiv Detail & Related papers (2026-02-06T09:22:24Z)
Tight Privacy Audit in One Run [14.266167758603986]
We show that our method achieves tight audit results for various differentially private protocols.<n>We also provide experiments that give contrasting conclusions to previous work on the parameter settings for privacy audits in one run.
arXiv Detail & Related papers (2025-09-10T15:55:03Z)
Privacy Auditing of Large Language Models [39.36184297797284]
We develop canaries that are far more effective than those used in prior work under threat models. For measuring the memorization rate of non-privately trained LLMs, our designed canaries surpass prior approaches.
arXiv Detail & Related papers (2025-03-09T23:32:15Z)
Privacy Audit as Bits Transmission: (Im)possibilities for Audit by One Run [7.850976675388593]
We introduce a unifying framework for privacy audits based on information-theoretic principles. We demystify the method of privacy audit by one run, identifying the conditions under which single-run audits are feasible or infeasible.
arXiv Detail & Related papers (2025-01-29T16:38:51Z)
Auditing $f$-Differential Privacy in One Run [43.34594422920125]
Empirical auditing has emerged as a means of catching some of the flaws in the implementation of privacy-preserving algorithms. We present a tight and efficient auditing procedure and analysis that can effectively assess the privacy of mechanisms.
arXiv Detail & Related papers (2024-10-29T17:02:22Z)
The Last Iterate Advantage: Empirical Auditing and Principled Heuristic Analysis of Differentially Private SGD [46.71175773861434]
We propose a simple privacy analysis of noisy clipped gradient descent (DP-SGD) We show experimentally that our is predictive of the outcome of privacy auditing applied to various training procedures. We also empirically support our and show existing privacy auditing attacks are bounded by our analysis in both vision and language tasks.
arXiv Detail & Related papers (2024-10-08T16:51:10Z)
Convergent Differential Privacy Analysis for General Federated Learning: the $f$-DP Perspective [57.35402286842029]
Federated learning (FL) is an efficient collaborative training paradigm with a focus on local privacy. differential privacy (DP) is a classical approach to capture and ensure the reliability of private protections.
arXiv Detail & Related papers (2024-08-28T08:22:21Z)
Nearly Tight Black-Box Auditing of Differentially Private Machine Learning [10.305660258428993]
This paper presents an auditing procedure for the Differentially Private Gradient Descent (DP-SGD) algorithm in the black-box threat model. The main intuition is to craft worst-case initial model parameters, as DP-SGD's privacy analysis is agnostic to the choice of the initial model parameters.
arXiv Detail & Related papers (2024-05-23T02:24:52Z)
Tight Auditing of Differentially Private Machine Learning [77.38590306275877]
For private machine learning, existing auditing mechanisms are tight. They only give tight estimates under implausible worst-case assumptions. We design an improved auditing scheme that yields tight privacy estimates for natural (not adversarially crafted) datasets.
arXiv Detail & Related papers (2023-02-15T21:40:33Z)
Analyzing Privacy Leakage in Machine Learning via Multiple Hypothesis Testing: A Lesson From Fano [83.5933307263932]
We study data reconstruction attacks for discrete data and analyze it under the framework of hypothesis testing. We show that if the underlying private data takes values from a set of size $M$, then the target privacy parameter $epsilon$ can be $O(log M)$ before the adversary gains significant inferential power.
arXiv Detail & Related papers (2022-10-24T23:50:12Z)
MaxMatch: Semi-Supervised Learning with Worst-Case Consistency [149.03760479533855]
We propose a worst-case consistency regularization technique for semi-supervised learning (SSL) We present a generalization bound for SSL consisting of the empirical loss terms observed on labeled and unlabeled training data separately. Motivated by this bound, we derive an SSL objective that minimizes the largest inconsistency between an original unlabeled sample and its multiple augmented variants.
arXiv Detail & Related papers (2022-09-26T12:04:49Z)
Connect the Dots: Tighter Discrete Approximations of Privacy Loss Distributions [49.726408540784334]
Key question in PLD-based accounting is how to approximate any (potentially continuous) PLD with a PLD over any specified discrete support. We show that our pessimistic estimate is the best possible among all pessimistic estimates.
arXiv Detail & Related papers (2022-07-10T04:25:02Z)
Individual Privacy Accounting for Differentially Private Stochastic Gradient Descent [69.14164921515949]
We characterize privacy guarantees for individual examples when releasing models trained by DP-SGD. We find that most examples enjoy stronger privacy guarantees than the worst-case bound. This implies groups that are underserved in terms of model utility simultaneously experience weaker privacy guarantees.
arXiv Detail & Related papers (2022-06-06T13:49:37Z)
Learning with User-Level Privacy [61.62978104304273]
We analyze algorithms to solve a range of learning tasks under user-level differential privacy constraints. Rather than guaranteeing only the privacy of individual samples, user-level DP protects a user's entire contribution. We derive an algorithm that privately answers a sequence of $K$ adaptively chosen queries with privacy cost proportional to $tau$, and apply it to solve the learning tasks we consider.
arXiv Detail & Related papers (2021-02-23T18:25:13Z)
Tight Differential Privacy for Discrete-Valued Mechanisms and for the Subsampled Gaussian Mechanism Using FFT [6.929834518749884]
We propose a numerical accountant for evaluating the tight $(varepsilon,delta)$-privacy loss for algorithms with discrete one dimensional output. We show that our approach allows decreasing noise variance up to 75 percent at equal privacy compared to existing bounds in the literature.
arXiv Detail & Related papers (2020-06-12T12:46:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.