Defense against Poisoning Attacks under Shuffle-DP

• URL: http://arxiv.org/abs/2605.00625v1
• Date: Fri, 01 May 2026 12:55:36 GMT
• Title: Defense against Poisoning Attacks under Shuffle-DP
• Abstract summary: Differential Privacy (DP) has become the gold standard for protecting individual privacy in data analytics.<n>We propose the first general defense framework for all union-preserving queries, capable of transforming any shuffle-DP protocol into a version resilient to poisoning attacks.<n>Compared to the original shuffle-DP protocol, it retains equivalent error in attack-free settings and incurs only a polylogarithmic increase in error when a constant number of attackers are present.
• Score: 44.41503870217386
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Differential Privacy (DP) has become the gold standard for protecting individual privacy in data analytics, and the shuffle-DP model has attracted significant attention from both academia and industry due to its favorable balance between privacy and utility. However, existing shuffle-DP protocols rely on a strong assumption: all users behave honestly. In real-world scenarios, adversarial users can exploit this vulnerability through poisoning attacks, compromising both privacy guarantees and the utility of analytical results. While defending against poisoning attacks in the shuffle-DP model has recently gained interest, existing solutions are limited to frequency estimation tasks. To address this issue, we propose the first general defense framework for all union-preserving queries, capable of transforming any shuffle-DP protocol into a version resilient to poisoning attacks. Beyond robust defense against poisoning attacks, our framework achieves high utility of analytical results. Compared to the original shuffle-DP protocol, it retains asymptotically equivalent error in attack-free settings and incurs only a polylogarithmic increase in error when a constant number of attackers are present. We demonstrate the generality of our framework on several common queries, including summation, frequency estimation, and range counting. Experimental results confirm that our approach effectively defends against poisoning attacks while maintaining strong utility and communication efficiency.

Related papers

• Robust Estimation of Sparse Numerical Vectors under Local Differential Privacy [61.848691894557426]
  Local differential privacy (LDP) protocols are vulnerable to poisoning attacks.<n>In this paper, we address the robust sparse vector mean estimation problem.<n>We propose a correction method based on a careful analysis that gives an exact expression of the bias.
  arXiv  Detail & Related papers  (2026-07-30T07:55:09Z)
• Predictability as a Fine-Grained Measure for Privacy [4.861418621297099]
  We introduce privacy via predictability, a fine-grained framework that explicitly incorporates the attacker's core knowledge.<n> predictability provides a finer-grained privacy metric tailored to specific sensitive information and specific attacker models.<n>Our approach is complementary to Differential privacy (DP) and can be used alongside DP to provide fine-grained privacy control.
  arXiv  Detail & Related papers  (2026-06-18T17:55:31Z)
• Fully Oblivious Differential Privacy for Frequency Estimation in the Augmented Shuffle Model with Trusted Processors [6.69087470775851]
  In shuffle model of DP (Differential Privacy), a shuffler randomly permutes users' data to achieve high accuracy and privacy.<n>Recent studies show that most existing shuffle protocols are vulnerable to collusion attacks by the data collector and users.<n>We introduce FODP (Fully Oblivious DP), which strengthens DP to prevent various TEE side-channel attacks.
  arXiv  Detail & Related papers  (2026-06-08T12:21:42Z)
• TwinGate: Stateful Defense against Decompositional Jailbreaks in Untraceable Traffic via Asymmetric Contrastive Learning [60.68349524623048]
  Decompositional jailbreaks pose a critical threat to large language models.<n>We present TwinGate, a stateful dual-encoder defense framework.<n>We construct a comprehensive dataset of over 3.62 million instructions spanning 8,600 distinct malicious intents.
  arXiv  Detail & Related papers  (2026-04-30T13:44:01Z)
• RAIN: Secure and Robust Aggregation under Shuffle Model of Differential Privacy [46.52109845749167]
  We present Robust Aggregation in Noise (RAIN), a framework that reconciles privacy, robustness, and verifiability under Shuffle-DP.<n>RAIN adopts sign-space aggregation to robustly measure update consistency and limit malicious influence under noise and anonymization.<n>We show that RAIN maintains strong privacy guarantees under Shuffle-DP and remains robust to poisoning attacks with negligible degradation in accuracy and convergence.
  arXiv  Detail & Related papers  (2026-03-03T15:41:54Z)
• Beyond the Worst Case: Extending Differential Privacy Guarantees to Realistic Adversaries [17.780319275883127]
  Differential Privacy is a family of definitions that bound the worst-case privacy leakage of a mechanism.<n>This work sheds light on what the worst-case guarantee of DP implies about the success of attackers that are more representative of real-world privacy risks.
  arXiv  Detail & Related papers  (2025-07-10T20:36:31Z)
• PPFPL: Cross-silo Privacy-preserving Federated Prototype Learning Against Data Poisoning Attacks [16.603029614862432]
  Privacy-Preserving Federated Learning (PPFL) enables multiple clients to collaboratively train models by submitting secreted model updates.<n>PPFL is vulnerable to data poisoning attacks due to its distributed training paradigm in cross-silo scenarios.<n>Existing solutions have struggled to improve the performance of PPFL under poisoned Non-Independent and Identically Distributed (Non-IID) data.
  arXiv  Detail & Related papers  (2025-04-04T05:05:24Z)
• Bayes-Nash Generative Privacy Against Membership Inference Attacks [24.330984323956173]
  We propose a game-theoretic framework modeling privacy protection as a Bayesian game between defender and attacker.<n>To address strategic complexity, we represent the defender's mixed strategy as a neural network generator mapping private datasets to public representations.<n>Our approach significantly outperforms state-of-the-art methods by generating stronger attacks and achieving better privacy-utility tradeoffs.
  arXiv  Detail & Related papers  (2024-10-09T20:29:04Z)
• Data Poisoning Attacks to Locally Differentially Private Frequent Itemset Mining Protocols [13.31395140464466]
  Local differential privacy (LDP) provides a way for an untrusted data collector to aggregate users' data without violating their privacy. Various privacy-preserving data analysis tasks have been studied under the protection of LDP, such as frequency estimation, frequent itemset mining, and machine learning. Recent research has demonstrated the vulnerability of certain LDP protocols to data poisoning attacks.
  arXiv  Detail & Related papers  (2024-06-27T18:11:19Z)
• From Mean to Extreme: Formal Differential Privacy Bounds on the Success of Real-World Data Reconstruction Attacks [54.25638567385662]
  Differential Privacy in machine learning is often interpreted as guarantees against membership inference.<n> translating DP budgets into quantitative protection against the more damaging threat of data reconstruction remains a challenging open problem.<n>This paper bridges the critical gap by deriving the first formal privacy bounds tailored to the mechanics of demonstrated "from-scratch" attacks.
  arXiv  Detail & Related papers  (2024-02-20T09:52:30Z)
• FreqFed: A Frequency Analysis-Based Approach for Mitigating Poisoning Attacks in Federated Learning [98.43475653490219]
  Federated learning (FL) is susceptible to poisoning attacks. FreqFed is a novel aggregation mechanism that transforms the model updates into the frequency domain. We demonstrate that FreqFed can mitigate poisoning attacks effectively with a negligible impact on the utility of the aggregated model.
  arXiv  Detail & Related papers  (2023-12-07T16:56:24Z)
• Is Vertical Logistic Regression Privacy-Preserving? A Comprehensive Privacy Analysis and Beyond [57.10914865054868]
  We consider vertical logistic regression (VLR) trained with mini-batch descent gradient. We provide a comprehensive and rigorous privacy analysis of VLR in a class of open-source Federated Learning frameworks.
  arXiv  Detail & Related papers  (2022-07-19T05:47:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.