Unveiling Privacy, Memorization, and Input Curvature Links
- URL: http://arxiv.org/abs/2402.18726v1
- Date: Wed, 28 Feb 2024 22:02:10 GMT
- Title: Unveiling Privacy, Memorization, and Input Curvature Links
- Authors: Deepak Ravikumar, Efstathia Soufleri, Abolfazl Hashemi, Kaushik Roy
- Abstract summary: Memorization is closely related to several concepts such as generalization, noisy learning, and privacy.
Recent research has shown evidence linking input loss curvature (measured by the trace of the loss Hessian w.r.t inputs) and memorization.
We extend our analysis to establish theoretical links between differential privacy, memorization, and input loss curvature.
- Score: 11.290935303784208
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Deep Neural Nets (DNNs) have become a pervasive tool for solving many
emerging problems. However, they tend to overfit to and memorize the training
set. Memorization is of keen interest since it is closely related to several
concepts such as generalization, noisy learning, and privacy. To study
memorization, Feldman (2019) proposed a formal score, however its computational
requirements limit its practical use. Recent research has shown empirical
evidence linking input loss curvature (measured by the trace of the loss
Hessian w.r.t inputs) and memorization. It was shown to be ~3 orders of
magnitude more efficient than calculating the memorization score. However,
there is a lack of theoretical understanding linking memorization with input
loss curvature. In this paper, we not only investigate this connection but also
extend our analysis to establish theoretical links between differential
privacy, memorization, and input loss curvature. First, we derive an upper
bound on memorization characterized by both differential privacy and input loss
curvature. Second, we present a novel insight showing that input loss curvature
is upper-bounded by the differential privacy parameter. Our theoretical
findings are further empirically validated using deep models on CIFAR and
ImageNet datasets, showing a strong correlation between our theoretical
predictions and results observed in practice.
Related papers
- Curvature Clues: Decoding Deep Learning Privacy with Input Loss Curvature [6.738409533239947]
Curvature of loss with respect to input (termed input loss curvature) is the trace of the Hessian of the loss with respect to the input.
We develop a theoretical framework that derives an upper bound on the train-test distinguishability based on privacy and the size of the training set.
This insight fuels the development of a new black box membership inference attack utilizing input loss curvature.
arXiv Detail & Related papers (2024-07-03T01:47:46Z) - On the Dynamics Under the Unhinged Loss and Beyond [104.49565602940699]
We introduce the unhinged loss, a concise loss function, that offers more mathematical opportunities to analyze closed-form dynamics.
The unhinged loss allows for considering more practical techniques, such as time-vary learning rates and feature normalization.
arXiv Detail & Related papers (2023-12-13T02:11:07Z) - How to Construct Perfect and Worse-than-Coin-Flip Spoofing
Countermeasures: A Word of Warning on Shortcut Learning [20.486639064376014]
Shortcut learning, or Clever Hans effect refers to situations where a learning agent learns spurious correlations present in data, resulting in biased models.
We focus on finding shortcuts in deep learning based spoofing countermeasures (CMs) that predict whether a given utterance is spoofed or not.
arXiv Detail & Related papers (2023-05-31T15:58:37Z) - Fundamental Limits of Two-layer Autoencoders, and Achieving Them with
Gradient Methods [91.54785981649228]
This paper focuses on non-linear two-layer autoencoders trained in the challenging proportional regime.
Our results characterize the minimizers of the population risk, and show that such minimizers are achieved by gradient methods.
For the special case of a sign activation function, our analysis establishes the fundamental limits for the lossy compression of Gaussian sources via (shallow) autoencoders.
arXiv Detail & Related papers (2022-12-27T12:37:34Z) - Mathematical Justification of Hard Negative Mining via Isometric
Approximation Theorem [18.525368151998386]
In deep metric learning, the Triplet Loss has emerged as a popular method to learn many computer vision and natural language processing tasks.
One issue that plagues the Triplet Loss is network collapse, an undesirable phenomenon where the network projects the embeddings of all data onto a single point.
While hard negative mining is the most effective of these strategies, existing formulations lack strong theoretical justification for their empirical success.
arXiv Detail & Related papers (2022-10-20T11:21:17Z) - Towards Differential Relational Privacy and its use in Question
Answering [109.4452196071872]
Memorization of relation between entities in a dataset can lead to privacy issues when using a trained question answering model.
We quantify this phenomenon and provide a possible definition of Differential Privacy (DPRP)
We illustrate concepts in experiments with largescale models for Question Answering.
arXiv Detail & Related papers (2022-03-30T22:59:24Z) - Towards an Understanding of Benign Overfitting in Neural Networks [104.2956323934544]
Modern machine learning models often employ a huge number of parameters and are typically optimized to have zero training loss.
We examine how these benign overfitting phenomena occur in a two-layer neural network setting.
We show that it is possible for the two-layer ReLU network interpolator to achieve a near minimax-optimal learning rate.
arXiv Detail & Related papers (2021-06-06T19:08:53Z) - Bounding Information Leakage in Machine Learning [26.64770573405079]
This paper investigates fundamental bounds on information leakage.
We identify and bound the success rate of the worst-case membership inference attack.
We derive bounds on the mutual information between the sensitive attributes and model parameters.
arXiv Detail & Related papers (2021-05-09T08:49:14Z) - Fundamental Limits and Tradeoffs in Invariant Representation Learning [99.2368462915979]
Many machine learning applications involve learning representations that achieve two competing goals.
Minimax game-theoretic formulation represents a fundamental tradeoff between accuracy and invariance.
We provide an information-theoretic analysis of this general and important problem under both classification and regression settings.
arXiv Detail & Related papers (2020-12-19T15:24:04Z) - Towards Certified Robustness of Distance Metric Learning [53.96113074344632]
We advocate imposing an adversarial margin in the input space so as to improve the generalization and robustness of metric learning algorithms.
We show that the enlarged margin is beneficial to the generalization ability by using the theoretical technique of algorithmic robustness.
arXiv Detail & Related papers (2020-06-10T16:51:53Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.