Distribution inference risks: Identifying and mitigating sources of leakage

• URL: http://arxiv.org/abs/2209.08541v1
• Date: Sun, 18 Sep 2022 11:45:27 GMT
• Title: Distribution inference risks: Identifying and mitigating sources of leakage
• Authors: Valentin Hartmann, L\'eo Meynent, Maxime Peyrard, Dimitrios Dimitriadis, Shruti Tople, Robert West
• Abstract summary: Leakage due to distribution inference (or property inference) attacks is gaining attention. In this attack, the goal of an adversary is to infer distributional information about the training data. We theoretically and empirically analyze the sources of information leakage that allows an adversary to perpetrate distribution inference attacks.
• Score: 42.1976069096515
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A large body of work shows that machine learning (ML) models can leak sensitive or confidential information about their training data. Recently, leakage due to distribution inference (or property inference) attacks is gaining attention. In this attack, the goal of an adversary is to infer distributional information about the training data. So far, research on distribution inference has focused on demonstrating successful attacks, with little attention given to identifying the potential causes of the leakage and to proposing mitigations. To bridge this gap, as our main contribution, we theoretically and empirically analyze the sources of information leakage that allows an adversary to perpetrate distribution inference attacks. We identify three sources of leakage: (1) memorizing specific information about the $\mathbb{E}[Y|X]$ (expected label given the feature values) of interest to the adversary, (2) wrong inductive bias of the model, and (3) finiteness of the training data. Next, based on our analysis, we propose principled mitigation techniques against distribution inference attacks. Specifically, we demonstrate that causal learning techniques are more resilient to a particular type of distribution inference risk termed distributional membership inference than associative learning methods. And lastly, we present a formalization of distribution inference that allows for reasoning about more general adversaries than was previously possible.

Related papers

• Extracting Training Data from Unconditional Diffusion Models [76.85077961718875]
  diffusion probabilistic models (DPMs) are being employed as mainstream models for generative artificial intelligence (AI) We aim to establish a theoretical understanding of memorization in DPMs with 1) a memorization metric for theoretical analysis, 2) an analysis of conditional memorization with informative and random labels, and 3) two better evaluation metrics for measuring memorization. Based on the theoretical analysis, we propose a novel data extraction method called textbfSurrogate condItional Data Extraction (SIDE) that leverages a trained on generated data as a surrogate condition to extract training data directly from unconditional diffusion models.
  arXiv  Detail & Related papers  (2024-06-18T16:20:12Z)
• Exploring Privacy and Fairness Risks in Sharing Diffusion Models: An Adversarial Perspective [31.010937126289953]
  We take an adversarial perspective to investigate the potential privacy and fairness risks associated with sharing of diffusion models. We demonstrate that the sharer can execute fairness poisoning attacks to undermine the receiver's downstream models. Our experiments conducted on real-world datasets demonstrate remarkable attack performance on different types of diffusion models.
  arXiv  Detail & Related papers  (2024-02-28T12:21:12Z)
• Fundamental Limits of Membership Inference Attacks on Machine Learning Models [29.367087890055995]
  Membership inference attacks (MIA) can reveal whether a particular data point was part of the training dataset, potentially exposing sensitive information about individuals. This article provides theoretical guarantees by exploring the fundamental statistical limitations associated with MIAs on machine learning models.
  arXiv  Detail & Related papers  (2023-10-20T19:32:54Z)
• Robust Transferable Feature Extractors: Learning to Defend Pre-Trained Networks Against White Box Adversaries [69.53730499849023]
  We show that adversarial examples can be successfully transferred to another independently trained model to induce prediction errors. We propose a deep learning-based pre-processing mechanism, which we refer to as a robust transferable feature extractor (RTFE)
  arXiv  Detail & Related papers  (2022-09-14T21:09:34Z)
• Formalizing and Estimating Distribution Inference Risks [11.650381752104298]
  We propose a formal and general definition of property inference attacks. Our results show that inexpensive attacks are as effective as expensive meta-classifier attacks. We extend the state-of-the-art property inference attack to work on convolutional neural networks.
  arXiv  Detail & Related papers  (2021-09-13T14:54:39Z)
• Adversarial Robustness through the Lens of Causality [105.51753064807014]
  adversarial vulnerability of deep neural networks has attracted significant attention in machine learning. We propose to incorporate causality into mitigating adversarial vulnerability. Our method can be seen as the first attempt to leverage causality for mitigating adversarial vulnerability.
  arXiv  Detail & Related papers  (2021-06-11T06:55:02Z)
• Formalizing Distribution Inference Risks [11.650381752104298]
  Property inference attacks are difficult to distinguish from the primary purposes of statistical machine learning. We propose a formal and generic definition of property inference attacks.
  arXiv  Detail & Related papers  (2021-06-07T15:10:06Z)
• Trust but Verify: Assigning Prediction Credibility by Counterfactual Constrained Learning [123.3472310767721]
  Prediction credibility measures are fundamental in statistics and machine learning. These measures should account for the wide variety of models used in practice. The framework developed in this work expresses the credibility as a risk-fit trade-off.
  arXiv  Detail & Related papers  (2020-11-24T19:52:38Z)
• Learning while Respecting Privacy and Robustness to Distributional Uncertainties and Adversarial Data [66.78671826743884]
  The distributionally robust optimization framework is considered for training a parametric model. The objective is to endow the trained model with robustness against adversarially manipulated input data. Proposed algorithms offer robustness with little overhead.
  arXiv  Detail & Related papers  (2020-07-07T18:25:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.