Unintended memorisation of unique features in neural networks

• URL: http://arxiv.org/abs/2205.10079v1
• Date: Fri, 20 May 2022 10:48:18 GMT
• Title: Unintended memorisation of unique features in neural networks
• Authors: John Hartley, Sotirios A. Tsaftaris
• Abstract summary: We show that unique features occurring only once in training data are memorised by discriminative multi-layer perceptrons and convolutional neural networks. We develop a score estimating a model's sensitivity to a unique feature by comparing the KL divergences of the model's output distributions. We find that typical strategies to prevent overfitting do not prevent unique feature memorisation.
• Score: 15.174895411434026
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neural networks pose a privacy risk due to their propensity to memorise and leak training data. We show that unique features occurring only once in training data are memorised by discriminative multi-layer perceptrons and convolutional neural networks trained on benchmark imaging datasets. We design our method for settings where sensitive training data is not available, for example medical imaging. Our setting knows the unique feature, but not the training data, model weights or the unique feature's label. We develop a score estimating a model's sensitivity to a unique feature by comparing the KL divergences of the model's output distributions given modified out-of-distribution images. We find that typical strategies to prevent overfitting do not prevent unique feature memorisation. And that images containing a unique feature are highly influential, regardless of the influence the images's other features. We also find a significant variation in memorisation with training seed. These results imply that neural networks pose a privacy risk to rarely occurring private information. This risk is more pronounced in healthcare applications since sensitive patient information can be memorised when it remains in training data due to an imperfect data sanitisation process.

Related papers

• Accurate Forgetting for All-in-One Image Restoration Model [3.367455972998532]
  Currently, a low-cost scheme called Machine Unlearning forgets the private data remembered in the model. Inspired by this, we try to use this concept to bridge the gap between the fields of image restoration and security.
  arXiv  Detail & Related papers  (2024-09-01T10:14:16Z)
• You Can Use But Cannot Recognize: Preserving Visual Privacy in Deep Neural Networks [29.03438707988713]
  Existing privacy protection techniques are unable to efficiently protect such data. We propose a novel privacy-preserving framework VisualMixer. VisualMixer shuffles pixels in the spatial domain and in the chromatic channel space in the regions without injecting noises. Experiments on real-world datasets demonstrate that VisualMixer can effectively preserve the visual privacy with negligible accuracy loss.
  arXiv  Detail & Related papers  (2024-04-05T13:49:27Z)
• Attribute-preserving Face Dataset Anonymization via Latent Code Optimization [64.4569739006591]
  We present a task-agnostic anonymization procedure that directly optimize the images' latent representation in the latent space of a pre-trained GAN. We demonstrate through a series of experiments that our method is capable of anonymizing the identity of the images whilst -- crucially -- better-preserving the facial attributes.
  arXiv  Detail & Related papers  (2023-03-20T17:34:05Z)
• Reconstructing Training Data from Model Gradient, Provably [68.21082086264555]
  We reconstruct the training samples from a single gradient query at a randomly chosen parameter value. As a provable attack that reveals sensitive training data, our findings suggest potential severe threats to privacy.
  arXiv  Detail & Related papers  (2022-12-07T15:32:22Z)
• Measuring Unintended Memorisation of Unique Private Features in Neural Networks [15.174895411434026]
  We show that neural networks unintentionally memorise unique features even when they occur only once in training data. An example of a unique feature is a person's name that is accidentally present on a training image.
  arXiv  Detail & Related papers  (2022-02-16T14:39:05Z)
• Syfer: Neural Obfuscation for Private Data Release [58.490998583666276]
  We develop Syfer, a neural obfuscation method to protect against re-identification attacks. Syfer composes trained layers with random neural networks to encode the original data. It maintains the ability to predict diagnoses from the encoded data.
  arXiv  Detail & Related papers  (2022-01-28T20:32:04Z)
• Robustness Threats of Differential Privacy [70.818129585404]
  We experimentally demonstrate that networks, trained with differential privacy, in some settings might be even more vulnerable in comparison to non-private versions. We study how the main ingredients of differentially private neural networks training, such as gradient clipping and noise addition, affect the robustness of the model.
  arXiv  Detail & Related papers  (2020-12-14T18:59:24Z)
• Encoding Robustness to Image Style via Adversarial Feature Perturbations [72.81911076841408]
  We adapt adversarial training by directly perturbing feature statistics, rather than image pixels, to produce robust models. Our proposed method, Adversarial Batch Normalization (AdvBN), is a single network layer that generates worst-case feature perturbations during training.
  arXiv  Detail & Related papers  (2020-09-18T17:52:34Z)
• Automatic Recall Machines: Internal Replay, Continual Learning and the Brain [104.38824285741248]
  Replay in neural networks involves training on sequential data with memorized samples, which counteracts forgetting of previous behavior caused by non-stationarity. We present a method where these auxiliary samples are generated on the fly, given only the model that is being trained for the assessed objective. Instead the implicit memory of learned samples within the assessed model itself is exploited.
  arXiv  Detail & Related papers  (2020-06-22T15:07:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.