Mitigating Data Poisoning Attacks to Local Differential Privacy

• URL: http://arxiv.org/abs/2506.02156v2
• Date: Mon, 16 Jun 2025 18:08:15 GMT
• Title: Mitigating Data Poisoning Attacks to Local Differential Privacy
• Authors: Xiaolin Li, Ninghui Li, Boyang Wang, Wenhai Sun,
• Abstract summary: We propose a comprehensive mitigation framework for popular frequency estimation, which contains a suite of novel defenses.<n>For detection, we present a new method to precisely identify bogus reports and thus LDP aggregation can be performed over the clean'' data.<n>When the attack behavior becomes stealthy and direct filtering out malicious users is difficult, we propose a detection that can effectively recognize hidden adversarial patterns.
• Score: 14.050238622718798
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The distributed nature of local differential privacy (LDP) invites data poisoning attacks and poses unforeseen threats to the underlying LDP-supported applications. In this paper, we propose a comprehensive mitigation framework for popular frequency estimation, which contains a suite of novel defenses, including malicious user detection, attack pattern recognition, and damaged utility recovery. In addition to existing attacks, we explore new adaptive adversarial activities for our mitigation design. For detection, we present a new method to precisely identify bogus reports and thus LDP aggregation can be performed over the ``clean'' data. When the attack behavior becomes stealthy and direct filtering out malicious users is difficult, we further propose a detection that can effectively recognize hidden adversarial patterns, thus facilitating the decision-making of service providers. These detection methods require no additional data and attack information and incur minimal computational cost. Our experiment demonstrates their excellent performance and substantial improvement over previous work in various settings. In addition, we conduct an empirical analysis of LDP post-processing for corrupted data recovery and propose a new post-processing method, through which we reveal new insights into protocol recommendations in practice and key design principles for future research.

Related papers

• DATABench: Evaluating Dataset Auditing in Deep Learning from an Adversarial Perspective [59.66984417026933]
  We introduce a novel taxonomy, classifying existing methods based on their reliance on internal features (IF) (inherent to the data) versus external features (EF) (artificially introduced for auditing)<n>We formulate two primary attack types: evasion attacks, designed to conceal the use of a dataset, and forgery attacks, intending to falsely implicate an unused dataset.<n>Building on the understanding of existing methods and attack objectives, we further propose systematic attack strategies: decoupling, removal, and detection for evasion; adversarial example-based methods for forgery.<n>Our benchmark, DATABench, comprises 17 evasion attacks, 5 forgery attacks, and 9
  arXiv  Detail & Related papers  (2025-07-08T03:07:15Z)
• Fine-grained Manipulation Attacks to Local Differential Privacy Protocols for Data Streams [19.89063520419922]
  Local Differential Privacy (LDP) enables massive data collection and analysis while protecting users' privacy.<n>Recent findings indicate that LDP protocols can be easily disrupted by poisoning or manipulation attacks.<n>Our research fills the gap by developing novel fine-grained manipulation attacks to LDP protocols for data streams.
  arXiv  Detail & Related papers  (2025-05-02T14:09:56Z)
• DataSentinel: A Game-Theoretic Detection of Prompt Injection Attacks [101.52204404377039]
  LLM-integrated applications and agents are vulnerable to prompt injection attacks.<n>A detection method aims to determine whether a given input is contaminated by an injected prompt.<n>We propose DataSentinel, a game-theoretic method to detect prompt injection attacks.
  arXiv  Detail & Related papers  (2025-04-15T16:26:21Z)
• Poisoning Attacks to Local Differential Privacy Protocols for Trajectory Data [14.934626547047763]
  Trajectory data, which tracks movements through geographic locations, is crucial for improving real-world applications.<n>Local differential privacy (LDP) offers a solution by allowing individuals to locally perturb their trajectory data before sharing it.<n>Despite its privacy benefits, LDP protocols are vulnerable to data poisoning attacks, where attackers inject fake data to manipulate aggregated results.
  arXiv  Detail & Related papers  (2025-03-06T02:31:45Z)
• Data Poisoning Attacks to Locally Differentially Private Range Query Protocols [15.664794320925562]
  Local Differential Privacy (LDP) has been widely adopted to protect user privacy in decentralized data collection.<n>Recent studies have revealed that LDP protocols are vulnerable to data poisoning attacks.<n>We present the first study on data poisoning attacks targeting LDP range query protocols.
  arXiv  Detail & Related papers  (2025-03-05T12:40:34Z)
• Hide in Plain Sight: Clean-Label Backdoor for Auditing Membership Inference [16.893873979953593]
  We propose a novel clean-label backdoor-based approach for stealthy data auditing. Our approach employs an optimal trigger generated by a shadow model that mimics target model's behavior. The proposed method enables robust data auditing through blackbox access, achieving high attack success rates across diverse datasets.
  arXiv  Detail & Related papers  (2024-11-24T20:56:18Z)
• On the Robustness of LDP Protocols for Numerical Attributes under Data Poisoning Attacks [17.351593328097977]
  Local differential privacy (LDP) protocols are vulnerable to data poisoning attacks.<n>This vulnerability raises concerns regarding the robustness and reliability of LDP in hostile environments.
  arXiv  Detail & Related papers  (2024-03-28T15:43:38Z)
• On the Universal Adversarial Perturbations for Efficient Data-free Adversarial Detection [55.73320979733527]
  We propose a data-agnostic adversarial detection framework, which induces different responses between normal and adversarial samples to UAPs. Experimental results show that our method achieves competitive detection performance on various text classification tasks.
  arXiv  Detail & Related papers  (2023-06-27T02:54:07Z)
• Avoid Adversarial Adaption in Federated Learning by Multi-Metric Investigations [55.2480439325792]
  Federated Learning (FL) facilitates decentralized machine learning model training, preserving data privacy, lowering communication costs, and boosting model performance through diversified data sources. FL faces vulnerabilities such as poisoning attacks, undermining model integrity with both untargeted performance degradation and targeted backdoor attacks. We define a new notion of strong adaptive adversaries, capable of adapting to multiple objectives simultaneously. MESAS is the first defense robust against strong adaptive adversaries, effective in real-world data scenarios, with an average overhead of just 24.37 seconds.
  arXiv  Detail & Related papers  (2023-06-06T11:44:42Z)
• Re-thinking Data Availablity Attacks Against Deep Neural Networks [53.64624167867274]
  In this paper, we re-examine the concept of unlearnable examples and discern that the existing robust error-minimizing noise presents an inaccurate optimization objective. We introduce a novel optimization paradigm that yields improved protection results with reduced computational time requirements.
  arXiv  Detail & Related papers  (2023-05-18T04:03:51Z)
• Balancing detectability and performance of attacks on the control channel of Markov Decision Processes [77.66954176188426]
  We investigate the problem of designing optimal stealthy poisoning attacks on the control channel of Markov decision processes (MDPs) This research is motivated by the recent interest of the research community for adversarial and poisoning attacks applied to MDPs, and reinforcement learning (RL) methods.
  arXiv  Detail & Related papers  (2021-09-15T09:13:10Z)
• Sampling Attacks: Amplification of Membership Inference Attacks by Repeated Queries [74.59376038272661]
  We introduce sampling attack, a novel membership inference technique that unlike other standard membership adversaries is able to work under severe restriction of no access to scores of the victim model. We show that a victim model that only publishes the labels is still susceptible to sampling attacks and the adversary can recover up to 100% of its performance. For defense, we choose differential privacy in the form of gradient perturbation during the training of the victim model as well as output perturbation at prediction time.
  arXiv  Detail & Related papers  (2020-09-01T12:54:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.