Curvature Clues: Decoding Deep Learning Privacy with Input Loss Curvature

• URL: http://arxiv.org/abs/2407.02747v1
• Date: Wed, 3 Jul 2024 01:47:46 GMT
• Title: Curvature Clues: Decoding Deep Learning Privacy with Input Loss Curvature
• Authors: Deepak Ravikumar, Efstathia Soufleri, Kaushik Roy,
• Abstract summary: Curvature of loss with respect to input (termed input loss curvature) is the trace of the Hessian of the loss with respect to the input. We develop a theoretical framework that derives an upper bound on the train-test distinguishability based on privacy and the size of the training set. This insight fuels the development of a new black box membership inference attack utilizing input loss curvature.
• Score: 6.738409533239947
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In this paper, we explore the properties of loss curvature with respect to input data in deep neural networks. Curvature of loss with respect to input (termed input loss curvature) is the trace of the Hessian of the loss with respect to the input. We investigate how input loss curvature varies between train and test sets, and its implications for train-test distinguishability. We develop a theoretical framework that derives an upper bound on the train-test distinguishability based on privacy and the size of the training set. This novel insight fuels the development of a new black box membership inference attack utilizing input loss curvature. We validate our theoretical findings through experiments in computer vision classification tasks, demonstrating that input loss curvature surpasses existing methods in membership inference effectiveness. Our analysis highlights how the performance of membership inference attack (MIA) methods varies with the size of the training set, showing that curvature-based MIA outperforms other methods on sufficiently large datasets. This condition is often met by real datasets, as demonstrated by our results on CIFAR10, CIFAR100, and ImageNet. These findings not only advance our understanding of deep neural network behavior but also improve the ability to test privacy-preserving techniques in machine learning.

Related papers

• Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data [38.44734564565478]
  We provide a theoretical understanding of adversarial examples and adversarial training algorithms from the perspective of feature learning theory. We show that the adversarial training method can provably strengthen the robust feature learning and suppress the non-robust feature learning.
  arXiv  Detail & Related papers  (2024-10-11T03:59:49Z)
• Unveiling Privacy, Memorization, and Input Curvature Links [11.290935303784208]
  Memorization is closely related to several concepts such as generalization, noisy learning, and privacy. Recent research has shown evidence linking input loss curvature (measured by the trace of the loss Hessian w.r.t inputs) and memorization. We extend our analysis to establish theoretical links between differential privacy, memorization, and input loss curvature.
  arXiv  Detail & Related papers  (2024-02-28T22:02:10Z)
• Mitigating Privacy Risk in Membership Inference by Convex-Concave Loss [16.399746814823025]
  Machine learning models are susceptible to membership inference attacks (MIAs), which aim to infer whether a sample is in the training set. Existing work utilizes gradient ascent to enlarge the loss variance of training data, alleviating the privacy risk. We propose a novel method -- Convex-Concave Loss, which enables a high variance of training loss distribution by gradient descent.
  arXiv  Detail & Related papers  (2024-02-08T07:14:17Z)
• Class Incremental Learning for Adversarial Robustness [17.06592851567578]
  Adrial training integrates adversarial examples during model training to enhance robustness. We observe that combining incremental learning with naive adversarial training easily leads to a loss of robustness. We propose the Flatness Preserving Distillation (FPD) loss that leverages the output difference between adversarial and clean examples.
  arXiv  Detail & Related papers  (2023-12-06T04:38:02Z)
• Deep Neural Networks Tend To Extrapolate Predictably [51.303814412294514]
  neural network predictions tend to be unpredictable and overconfident when faced with out-of-distribution (OOD) inputs. We observe that neural network predictions often tend towards a constant value as input data becomes increasingly OOD. We show how one can leverage our insights in practice to enable risk-sensitive decision-making in the presence of OOD inputs.
  arXiv  Detail & Related papers  (2023-10-02T03:25:32Z)
• Understanding and Mitigating the Label Noise in Pre-training on Downstream Tasks [91.15120211190519]
  This paper aims to understand the nature of noise in pre-training datasets and to mitigate its impact on downstream tasks. We propose a light-weight black-box tuning method (NMTune) to affine the feature space to mitigate the malignant effect of noise.
  arXiv  Detail & Related papers  (2023-09-29T06:18:15Z)
• Delving into Sample Loss Curve to Embrace Noisy and Imbalanced Data [17.7825114228313]
  Corrupted labels and class imbalance are commonly encountered in practically collected training data. Existing approaches alleviate these issues by adopting a sample re-weighting strategy. However, biased samples with corrupted labels and of tailed classes commonly co-exist in training data.
  arXiv  Detail & Related papers  (2021-12-30T09:20:07Z)
• Attribute-Guided Adversarial Training for Robustness to Natural Perturbations [64.35805267250682]
  We propose an adversarial training approach which learns to generate new samples so as to maximize exposure of the classifier to the attributes-space. Our approach enables deep neural networks to be robust against a wide range of naturally occurring perturbations.
  arXiv  Detail & Related papers  (2020-12-03T10:17:30Z)
• Feature Purification: How Adversarial Training Performs Robust Deep Learning [66.05472746340142]
  We show a principle that we call Feature Purification, where we show one of the causes of the existence of adversarial examples is the accumulation of certain small dense mixtures in the hidden weights during the training process of a neural network. We present both experiments on the CIFAR-10 dataset to illustrate this principle, and a theoretical result proving that for certain natural classification tasks, training a two-layer neural network with ReLU activation using randomly gradient descent indeed this principle.
  arXiv  Detail & Related papers  (2020-05-20T16:56:08Z)
• Overfitting in adversarially robust deep learning [86.11788847990783]
  We show that overfitting to the training set does in fact harm robust performance to a very large degree in adversarially robust training. We also show that effects such as the double descent curve do still occur in adversarially trained models, yet fail to explain the observed overfitting.
  arXiv  Detail & Related papers  (2020-02-26T15:40:50Z)
• Identifying and Compensating for Feature Deviation in Imbalanced Deep Learning [59.65752299209042]
  We investigate learning a ConvNet under such a scenario. We found that a ConvNet significantly over-fits the minor classes. We propose to incorporate class-dependent temperatures (CDT) training ConvNet.
  arXiv  Detail & Related papers  (2020-01-06T03:52:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.