Reducing information dependency does not cause training data privacy. Adversarially non-robust features do

• URL: http://arxiv.org/abs/2607.12354v1
• Date: Tue, 14 Jul 2026 05:06:24 GMT
• Title: Reducing information dependency does not cause training data privacy. Adversarially non-robust features do
• Abstract summary: We show that extensive exposure can persist without rote memorization and is instead caused by a tunable connection to adversarial robustness.<n>Our results revise the prevailing understanding of training data exposure and reveal a new privacy-robustness tradeoff.
• Score: 1.9754011041953694
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we challenge the prevailing view that information dependency (including rote memorization) drives training data exposure to image reconstruction attacks. We show that extensive exposure can persist without rote memorization and is instead caused by a tunable connection to adversarial robustness. We begin by presenting three surprising results: (1) recent defenses that inhibit reconstruction by Model Inversion Attacks (MIAs), which evaluate leakage under an idealized attacker, do not reduce standard measures of information dependency (HSIC); (2) models that maximally memorize their training datasets remain robust to MIA reconstruction; and (3) models trained without seeing 97% of the training pixels, where recent information-theoretic bounds give arbitrarily strong privacy guarantees under standard assumptions, can still be devastatingly reconstructed by MIA. To explain these findings, we provide causal evidence that privacy under MIA arises from what the adversarial examples literature calls ``non-robust'' features (generalizable but imperceptible and unstable features). We further show that recent MIA defenses obtain their privacy improvements by unintentionally shifting models toward such features. To establish this causal relationship, we introduce Anti Adversarial Training (AT-AT), a training regime that intentionally learns non-robust features to obtain both superior reconstruction defense and higher accuracy than state-of-the-art defenses. Our results revise the prevailing understanding of training data exposure and reveal a new privacy-robustness tradeoff.

Related papers

• ARMOR: Manifold-Oriented Training for Adversarially Robust Aerial Object Detection under Data Scarcity [29.552503716131568]
  We propose Adversarial Robustness with Manifold-Oriented Training (ARMOR) to tackle model vulnerability and training data scarcity.<n>ARMOR realizes the core insights of on-manifold adversarial training (OMAT) in low-data regimes.<n>ARMOR maintains strong clean performance of over 0.90 model confidence, while improving adversarial robustness by up to 0.32 in model confidence over state-of-the-art defenses.
  arXiv  Detail & Related papers  (2026-08-30T02:14:40Z)
• No Prior, No Leakage: Revisiting Reconstruction Attacks in Trained Neural Networks [13.146179839752618]
  Training data by neural networks raises pressing concerns for privacy and security.<n>Recent work has shown that, under certain conditions, portions of the training set can be reconstructed directly from model parameters.<n>We analyze the inherent weaknesses and limitations of existing reconstruction methods and identify conditions under which they fail.
  arXiv  Detail & Related papers  (2025-09-25T15:14:08Z)
• Random Erasing vs. Model Inversion: A Promising Defense or a False Hope? [25.265902142904373]
  Model Inversion (MI) attacks pose a significant privacy threat by reconstructing private training data from machine learning models.<n>We explore Random Erasing (RE) to uncover its surprising effectiveness as a defense against MI attacks.<n>Our findings highlight RE as a simple yet effective defense mechanism that can be easily integrated with existing privacy-preserving techniques.
  arXiv  Detail & Related papers  (2024-09-02T08:37:17Z)
• Unlearning Backdoor Threats: Enhancing Backdoor Defense in Multimodal Contrastive Learning via Local Token Unlearning [49.242828934501986]
  Multimodal contrastive learning has emerged as a powerful paradigm for building high-quality features. backdoor attacks subtly embed malicious behaviors within the model during training. We introduce an innovative token-based localized forgetting training regime.
  arXiv  Detail & Related papers  (2024-03-24T18:33:15Z)
• Visual Privacy Auditing with Diffusion Models [47.0700328585184]
  We introduce a reconstruction attack based on diffusion models (DMs) that only assumes adversary access to real-world image priors.<n>We find that (1) real-world data priors significantly influence reconstruction success, (2) current reconstruction bounds do not model the risk posed by data priors well, and DMs can serve as auditing tools for visualizing privacy leakage.
  arXiv  Detail & Related papers  (2024-03-12T12:18:55Z)
• Reconciling AI Performance and Data Reconstruction Resilience for Medical Imaging [52.578054703818125]
  Artificial Intelligence (AI) models are vulnerable to information leakage of their training data, which can be highly sensitive. Differential Privacy (DP) aims to circumvent these susceptibilities by setting a quantifiable privacy budget. We show that using very large privacy budgets can render reconstruction attacks impossible, while drops in performance are negligible.
  arXiv  Detail & Related papers  (2023-12-05T12:21:30Z)
• Avoid Adversarial Adaption in Federated Learning by Multi-Metric Investigations [55.2480439325792]
  Federated Learning (FL) facilitates decentralized machine learning model training, preserving data privacy, lowering communication costs, and boosting model performance through diversified data sources. FL faces vulnerabilities such as poisoning attacks, undermining model integrity with both untargeted performance degradation and targeted backdoor attacks. We define a new notion of strong adaptive adversaries, capable of adapting to multiple objectives simultaneously. MESAS is the first defense robust against strong adaptive adversaries, effective in real-world data scenarios, with an average overhead of just 24.37 seconds.
  arXiv  Detail & Related papers  (2023-06-06T11:44:42Z)
• Re-thinking Data Availablity Attacks Against Deep Neural Networks [53.64624167867274]
  In this paper, we re-examine the concept of unlearnable examples and discern that the existing robust error-minimizing noise presents an inaccurate optimization objective. We introduce a novel optimization paradigm that yields improved protection results with reduced computational time requirements.
  arXiv  Detail & Related papers  (2023-05-18T04:03:51Z)
• On the Privacy Effect of Data Enhancement via the Lens of Memorization [20.63044895680223]
  We propose to investigate privacy from a new perspective called memorization. Through the lens of memorization, we find that previously deployed MIAs produce misleading results as they are less likely to identify samples with higher privacy risks. We demonstrate that the generalization gap and privacy leakage are less correlated than those of the previous results.
  arXiv  Detail & Related papers  (2022-08-17T13:02:17Z)
• RelaxLoss: Defending Membership Inference Attacks without Losing Utility [68.48117818874155]
  We propose a novel training framework based on a relaxed loss with a more achievable learning target. RelaxLoss is applicable to any classification model with added benefits of easy implementation and negligible overhead. Our approach consistently outperforms state-of-the-art defense mechanisms in terms of resilience against MIAs.
  arXiv  Detail & Related papers  (2022-07-12T19:34:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.