Bayes' capacity as a measure for reconstruction attacks in federated learning

• URL: http://arxiv.org/abs/2406.13569v1
• Date: Wed, 19 Jun 2024 13:58:42 GMT
• Title: Bayes' capacity as a measure for reconstruction attacks in federated learning
• Authors: Sayan Biswas, Mark Dras, Pedro Faustini, Natasha Fernandes, Annabelle McIver, Catuscia Palamidessi, Parastoo Sadeghi,
• Abstract summary: We formalise the reconstruction threat model using the information-theoretic framework of quantitative information flow. We show that the Bayes' capacity, related to the Sibson mutual information of order infinity, represents a tight upper bound on the leakage of the DP-SGD algorithm to an adversary.
• Score: 10.466570297146953
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Within the machine learning community, reconstruction attacks are a principal attack of concern and have been identified even in federated learning, which was designed with privacy preservation in mind. In federated learning, it has been shown that an adversary with knowledge of the machine learning architecture is able to infer the exact value of a training element given an observation of the weight updates performed during stochastic gradient descent. In response to these threats, the privacy community recommends the use of differential privacy in the stochastic gradient descent algorithm, termed DP-SGD. However, DP has not yet been formally established as an effective countermeasure against reconstruction attacks. In this paper, we formalise the reconstruction threat model using the information-theoretic framework of quantitative information flow. We show that the Bayes' capacity, related to the Sibson mutual information of order infinity, represents a tight upper bound on the leakage of the DP-SGD algorithm to an adversary interested in performing a reconstruction attack. We provide empirical results demonstrating the effectiveness of this measure for comparing mechanisms against reconstruction threats.

Related papers

• FEDLAD: Federated Evaluation of Deep Leakage Attacks and Defenses [50.921333548391345]
  Federated Learning is a privacy preserving decentralized machine learning paradigm. Recent research has revealed that private ground truth data can be recovered through a gradient technique known as Deep Leakage. This paper introduces the FEDLAD Framework (Federated Evaluation of Deep Leakage Attacks and Defenses), a comprehensive benchmark for evaluating Deep Leakage attacks and defenses.
  arXiv  Detail & Related papers  (2024-11-05T11:42:26Z)
• Enhancing Privacy of Spatiotemporal Federated Learning against Gradient Inversion Attacks [30.785476975412482]
  We propose Stemporal Gradient Inversion Attack (GIA), a gradient attack algorithm tailored totemporal data. We design an adaptive defense strategy to mitigate gradient inversion attacks intemporal federated learning. We reveal that the proposed defense strategy can well preserve the utility oftemporal federated learning with effective security protection.
  arXiv  Detail & Related papers  (2024-07-11T14:17:02Z)
• Unlearning Backdoor Threats: Enhancing Backdoor Defense in Multimodal Contrastive Learning via Local Token Unlearning [49.242828934501986]
  Multimodal contrastive learning has emerged as a powerful paradigm for building high-quality features. backdoor attacks subtly embed malicious behaviors within the model during training. We introduce an innovative token-based localized forgetting training regime.
  arXiv  Detail & Related papers  (2024-03-24T18:33:15Z)
• Improving Robustness to Model Inversion Attacks via Sparse Coding Architectures [4.962316236417777]
  Recent model inversion attack algorithms permit adversaries to reconstruct a neural network's private and potentially sensitive training data by repeatedly querying the network. We develop a novel network architecture that leverages sparse-coding layers to obtain superior robustness to this class of attacks.
  arXiv  Detail & Related papers  (2024-03-21T18:26:23Z)
• Visual Privacy Auditing with Diffusion Models [52.866433097406656]
  We propose a reconstruction attack based on diffusion models (DMs) that assumes adversary access to real-world image priors. We show that (1) real-world data priors significantly influence reconstruction success, (2) current reconstruction bounds do not model the risk posed by data priors well, and (3) DMs can serve as effective auditing tools for visualizing privacy leakage.
  arXiv  Detail & Related papers  (2024-03-12T12:18:55Z)
• Bounding Reconstruction Attack Success of Adversaries Without Data Priors [53.41619942066895]
  Reconstruction attacks on machine learning (ML) models pose a strong risk of leakage of sensitive data. In this work, we provide formal upper bounds on reconstruction success under realistic adversarial settings.
  arXiv  Detail & Related papers  (2024-02-20T09:52:30Z)
• Data Reconstruction Attacks and Defenses: A Systematic Evaluation [27.34562026045369]
  We propose to view the problem as an inverse problem, enabling us to theoretically, quantitatively, and systematically evaluate the data reconstruction problem. We propose a strong reconstruction attack that helps update some previous understanding of the strength of defense methods under our proposed evaluation metric.
  arXiv  Detail & Related papers  (2024-02-13T05:06:34Z)
• Reconstructing Training Data with Informed Adversaries [30.138217209991826]
  Given access to a machine learning model, can an adversary reconstruct the model's training data? This work studies this question from the lens of a powerful informed adversary who knows all the training data points except one. We show it is feasible to reconstruct the remaining data point in this stringent threat model.
  arXiv  Detail & Related papers  (2022-01-13T09:19:25Z)
• Learning to Learn Transferable Attack [77.67399621530052]
  Transfer adversarial attack is a non-trivial black-box adversarial attack that aims to craft adversarial perturbations on the surrogate model and then apply such perturbations to the victim model. We propose a Learning to Learn Transferable Attack (LLTA) method, which makes the adversarial perturbations more generalized via learning from both data and model augmentation. Empirical results on the widely-used dataset demonstrate the effectiveness of our attack method with a 12.85% higher success rate of transfer attack compared with the state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-12-10T07:24:21Z)
• Where Did You Learn That From? Surprising Effectiveness of Membership Inference Attacks Against Temporally Correlated Data in Deep Reinforcement Learning [114.9857000195174]
  A major challenge to widespread industrial adoption of deep reinforcement learning is the potential vulnerability to privacy breaches. We propose an adversarial attack framework tailored for testing the vulnerability of deep reinforcement learning algorithms to membership inference attacks.
  arXiv  Detail & Related papers  (2021-09-08T23:44:57Z)
• Exploring the Security Boundary of Data Reconstruction via Neuron Exclusivity Analysis [23.07323180340961]
  We study the security boundary of data reconstruction from gradient via a microcosmic view on neural networks with rectified linear units (ReLUs) We construct a novel deterministic attack algorithm which substantially outperforms previous attacks for reconstructing training batches lying in the insecure boundary of a neural network.
  arXiv  Detail & Related papers  (2020-10-26T05:54:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.