Disparate Privacy Vulnerability: Targeted Attribute Inference Attacks and Defenses

• URL: http://arxiv.org/abs/2504.04033v1
• Date: Sat, 05 Apr 2025 02:58:37 GMT
• Title: Disparate Privacy Vulnerability: Targeted Attribute Inference Attacks and Defenses
• Authors: Ehsanul Kabir, Lucas Craig, Shagufta Mehnaz,
• Abstract summary: A potential threat arises from an adversary querying trained models using the public, non-sensitive attributes of entities in the training data.<n>We develop a novel inference attack called the disparity inference attack, which targets the identification of high-risk groups within the dataset.<n>We are also the first to introduce a novel and effective disparity mitigation technique that simultaneously preserves model performance and prevents any risk of targeted attacks.
• Score: 1.740992908651449
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As machine learning (ML) technologies become more prevalent in privacy-sensitive areas like healthcare and finance, eventually incorporating sensitive information in building data-driven algorithms, it is vital to scrutinize whether these data face any privacy leakage risks. One potential threat arises from an adversary querying trained models using the public, non-sensitive attributes of entities in the training data to infer their private, sensitive attributes, a technique known as the attribute inference attack. This attack is particularly deceptive because, while it may perform poorly in predicting sensitive attributes across the entire dataset, it excels at predicting the sensitive attributes of records from a few vulnerable groups, a phenomenon known as disparate vulnerability. This paper illustrates that an adversary can take advantage of this disparity to carry out a series of new attacks, showcasing a threat level beyond previous imagination. We first develop a novel inference attack called the disparity inference attack, which targets the identification of high-risk groups within the dataset. We then introduce two targeted variations of the attribute inference attack that can identify and exploit a vulnerable subset of the training data, marking the first instances of targeted attacks in this category, achieving significantly higher accuracy than untargeted versions. We are also the first to introduce a novel and effective disparity mitigation technique that simultaneously preserves model performance and prevents any risk of targeted attacks.

Related papers

• When Machine Learning Models Leak: An Exploration of Synthetic Training Data [0.0]
  We investigate an attack on a machine learning model that predicts whether a person or household will relocate in the next two years. The attack assumes that the attacker can query the model to obtain predictions and that the marginal distribution of the data on which the model was trained is publicly available. We explore how replacing the original data with synthetic data when training the model impacts how successfully the attacker can infer sensitive attributes.
  arXiv  Detail & Related papers  (2023-10-12T23:47:22Z)
• Investigating Human-Identifiable Features Hidden in Adversarial Perturbations [54.39726653562144]
  Our study explores up to five attack algorithms across three datasets. We identify human-identifiable features in adversarial perturbations. Using pixel-level annotations, we extract such features and demonstrate their ability to compromise target models.
  arXiv  Detail & Related papers  (2023-09-28T22:31:29Z)
• Avoid Adversarial Adaption in Federated Learning by Multi-Metric Investigations [55.2480439325792]
  Federated Learning (FL) facilitates decentralized machine learning model training, preserving data privacy, lowering communication costs, and boosting model performance through diversified data sources. FL faces vulnerabilities such as poisoning attacks, undermining model integrity with both untargeted performance degradation and targeted backdoor attacks. We define a new notion of strong adaptive adversaries, capable of adapting to multiple objectives simultaneously. MESAS is the first defense robust against strong adaptive adversaries, effective in real-world data scenarios, with an average overhead of just 24.37 seconds.
  arXiv  Detail & Related papers  (2023-06-06T11:44:42Z)
• It Is All About Data: A Survey on the Effects of Data on Adversarial Robustness [4.1310970179750015]
  Adversarial examples are inputs to machine learning models that an attacker has intentionally designed to confuse the model into making a mistake. To address this problem, the area of adversarial robustness investigates mechanisms behind adversarial attacks and defenses against these attacks.
  arXiv  Detail & Related papers  (2023-03-17T04:18:03Z)
• On the Alignment of Group Fairness with Attribute Privacy [1.6574413179773757]
  Group fairness and privacy are fundamental aspects in designing trustworthy machine learning models. We are the first to demonstrate the alignment of group fairness with the specific privacy notion of attribute privacy in a blackbox setting.
  arXiv  Detail & Related papers  (2022-11-18T13:00:34Z)
• Adv-Attribute: Inconspicuous and Transferable Adversarial Attack on Face Recognition [111.1952945740271]
  Adversarial Attributes (Adv-Attribute) is designed to generate inconspicuous and transferable attacks on face recognition. Experiments on the FFHQ and CelebA-HQ datasets show that the proposed Adv-Attribute method achieves the state-of-the-art attacking success rates.
  arXiv  Detail & Related papers  (2022-10-13T09:56:36Z)
• Truth Serum: Poisoning Machine Learning Models to Reveal Their Secrets [53.866927712193416]
  We show that an adversary who can poison a training dataset can cause models trained on this dataset to leak private details belonging to other parties. Our attacks are effective across membership inference, attribute inference, and data extraction. Our results cast doubts on the relevance of cryptographic privacy guarantees in multiparty protocols for machine learning.
  arXiv  Detail & Related papers  (2022-03-31T18:06:28Z)
• Are Your Sensitive Attributes Private? Novel Model Inversion Attribute Inference Attacks on Classification Models [22.569705869469814]
  We focus on model inversion attacks where the adversary knows non-sensitive attributes about records in the training data. We devise a novel confidence score-based model inversion attribute inference attack that significantly outperforms the state-of-the-art. We also extend our attacks to the scenario where some of the other (non-sensitive) attributes of a target record are unknown to the adversary.
  arXiv  Detail & Related papers  (2022-01-23T21:27:20Z)
• Curse or Redemption? How Data Heterogeneity Affects the Robustness of Federated Learning [51.15273664903583]
  Data heterogeneity has been identified as one of the key features in federated learning but often overlooked in the lens of robustness to adversarial attacks. This paper focuses on characterizing and understanding its impact on backdooring attacks in federated learning through comprehensive experiments using synthetic and the LEAF benchmarks.
  arXiv  Detail & Related papers  (2021-02-01T06:06:21Z)
• Black-box Model Inversion Attribute Inference Attacks on Classification Models [32.757792981935815]
  We focus on one kind of model inversion attacks, where the adversary knows non-sensitive attributes about instances in the training data. We devise two novel model inversion attribute inference attacks -- confidence modeling-based attack and confidence score-based attack. We evaluate our attacks on two types of machine learning models, decision tree and deep neural network, trained with two real datasets.
  arXiv  Detail & Related papers  (2020-12-07T01:14:19Z)
• Sampling Attacks: Amplification of Membership Inference Attacks by Repeated Queries [74.59376038272661]
  We introduce sampling attack, a novel membership inference technique that unlike other standard membership adversaries is able to work under severe restriction of no access to scores of the victim model. We show that a victim model that only publishes the labels is still susceptible to sampling attacks and the adversary can recover up to 100% of its performance. For defense, we choose differential privacy in the form of gradient perturbation during the training of the victim model as well as output perturbation at prediction time.
  arXiv  Detail & Related papers  (2020-09-01T12:54:54Z)
• Systematic Evaluation of Privacy Risks of Machine Learning Models [41.017707772150835]
  We show that prior work on membership inference attacks may severely underestimate the privacy risks. We first propose to benchmark membership inference privacy risks by improving existing non-neural network based inference attacks. We then introduce a new approach for fine-grained privacy analysis by formulating and deriving a new metric called the privacy risk score.
  arXiv  Detail & Related papers  (2020-03-24T00:53:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.