Privacy Vulnerabilities of Attention Layers in Tabular Foundation Models and Protection of High-Risk Queries

• URL: http://arxiv.org/abs/2606.26021v1
• Date: Wed, 24 Jun 2026 16:42:21 GMT
• Title: Privacy Vulnerabilities of Attention Layers in Tabular Foundation Models and Protection of High-Risk Queries
• Abstract summary: In this paper, we demonstrate that predictions generated via the attention mechanism leak sufficient information to enable effective Membership Inference Attacks (MIAs)<n>We introduce an inference-time defence inspired by $k$-anonymity principles to mitigate this risk.
• Score: 12.545534835317481
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Tabular foundation models are commonly assumed to present limited privacy concerns as they are often pre-trained on large collections of synthetic data. However, these models leverage in-context learning, where sensitive records may be provided directly at inference time as labelled context examples. In this paper, we demonstrate that predictions generated via the attention mechanism leak sufficient information to enable effective Membership Inference Attacks (MIAs). To highlight this vulnerability, we propose AMIA (Attention-based Membership Inference Attack), a shadow-model-free attack that exploits the concentration of transformer attention patterns. Our results show that attention mechanisms reveal strong membership signals, which exceed classical confidence-based attacks, achieving an average gain of 7.7\%, specially in low false-positive regimes. To mitigate this risk, we introduce an inference-time defence inspired by $k$-anonymity principles. This approach reduces the uniqueness of context-key representations without introducing random noise or retraining the model. By targeting only high-risk queries identified through AMIA scores, the defence substantially reduces membership leakage of this attack by an average of 50\% and 25\% against confidence-based attacks, while preserving predictive utility with only 3.9\% performance degradation. Beyond showing that context examples are vulnerable, we further demonstrate that fine-tuning introduces an additional source of privacy risk. In particular, samples whose prediction confidence increases after fine-tuning become more susceptible to MIAs, indicating that fine-tuning can amplify memorisation and expose sensitive training information through confidence shifts.

Related papers

• Boundary-targeted Membership Inference Attacks on Safety Classifiers [70.20833439671131]
  Safety classifiers are trained on sensitive datasets including discussions of self-harm and mental health.<n>We introduce a new boundary-targeted selection strategy that identifies low confidence examples.<n>Experiments show that an adversary can recover 19% of the conversations a safety classifier flagged as indicating user distress, at a 5% false-positive rate.<n>This is $3.5$ times more than attacking using state-of-the-art MIA methods alone.
  arXiv  Detail & Related papers  (2026-05-21T12:05:22Z)
• Leveraging Soft Prompts for Privacy Attacks in Federated Prompt Tuning [24.914116986408327]
  We propose PromptMIA, a membership inference attack tailored to federated prompt-tuning.<n>We empirically show that PromptMIA consistently attains high advantage in this game across diverse benchmark datasets.<n>The results highlight non-trivial challenges for current defenses and offer insights into their limitations.
  arXiv  Detail & Related papers  (2026-01-10T17:50:05Z)
• VoxGuard: Evaluating User and Attribute Privacy in Speech via Membership Inference Attacks [51.68795949691009]
  We introduce VoxGuard, a framework grounded in differential privacy and membership inference.<n>For attributes, we show that simple transparent attacks recover gender and accent with near-perfect accuracy even after anonymization.<n>Our results demonstrate that EER substantially underestimates leakage, highlighting the need for low-FPR evaluation.
  arXiv  Detail & Related papers  (2025-09-22T20:57:48Z)
• Confidential Guardian: Cryptographically Prohibiting the Abuse of Model Abstention [65.47632669243657]
  A dishonest institution can exploit mechanisms to discriminate or unjustly deny services under the guise of uncertainty.<n>We demonstrate the practicality of this threat by introducing an uncertainty-inducing attack called Mirage.<n>We propose Confidential Guardian, a framework that analyzes calibration metrics on a reference dataset to detect artificially suppressed confidence.
  arXiv  Detail & Related papers  (2025-05-29T19:47:50Z)
• Unveiling Impact of Frequency Components on Membership Inference Attacks for Diffusion Models [51.179816451161635]
  Membership Inference Attacks (MIAs) are designed to ascertain whether specific data were utilized during a model's training phase.<n>We formalize them into a unified general paradigm which computes the membership score for membership identification.<n>Under this paradigm, we empirically find that existing attacks overlook the inherent deficiency in how diffusion models process high-frequency information.<n>We propose a plug-and-play high-frequency filter module to mitigate the adverse effects of the deficiency.
  arXiv  Detail & Related papers  (2025-05-27T09:50:11Z)
• Disparate Privacy Vulnerability: Targeted Attribute Inference Attacks and Defenses [1.740992908651449]
  A potential threat arises from an adversary querying trained models using the public, non-sensitive attributes of entities in the training data.<n>We develop a novel inference attack called the disparity inference attack, which targets the identification of high-risk groups within the dataset.<n>We are also the first to introduce a novel and effective disparity mitigation technique that simultaneously preserves model performance and prevents any risk of targeted attacks.
  arXiv  Detail & Related papers  (2025-04-05T02:58:37Z)
• Do Fairness Interventions Come at the Cost of Privacy: Evaluations for Binary Classifiers [17.243744418309593]
  We assess the privacy risks of fairness-enhanced binary classifiers via membership inference attacks (MIAs) and attribute inference attacks (AIAs)<n>We uncover a potential threat mechanism that exploits prediction discrepancies between fair and biased models, leading to advanced attack results for both MIAs and AIAs.<n>Our study exposes the under-explored privacy threats in fairness studies, advocating for thorough evaluations of potential security vulnerabilities before model deployments.
  arXiv  Detail & Related papers  (2025-03-08T10:21:21Z)
• Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Models [112.48136829374741]
  In this paper, we unveil a new vulnerability: the privacy backdoor attack. When a victim fine-tunes a backdoored model, their training data will be leaked at a significantly higher rate than if they had fine-tuned a typical model. Our findings highlight a critical privacy concern within the machine learning community and call for a reevaluation of safety protocols in the use of open-source pre-trained models.
  arXiv  Detail & Related papers  (2024-04-01T16:50:54Z)
• From Mean to Extreme: Formal Differential Privacy Bounds on the Success of Real-World Data Reconstruction Attacks [54.25638567385662]
  Differential Privacy in machine learning is often interpreted as guarantees against membership inference.<n> translating DP budgets into quantitative protection against the more damaging threat of data reconstruction remains a challenging open problem.<n>This paper bridges the critical gap by deriving the first formal privacy bounds tailored to the mechanics of demonstrated "from-scratch" attacks.
  arXiv  Detail & Related papers  (2024-02-20T09:52:30Z)
• Systematic Evaluation of Privacy Risks of Machine Learning Models [41.017707772150835]
  We show that prior work on membership inference attacks may severely underestimate the privacy risks. We first propose to benchmark membership inference privacy risks by improving existing non-neural network based inference attacks. We then introduce a new approach for fine-grained privacy analysis by formulating and deriving a new metric called the privacy risk score.
  arXiv  Detail & Related papers  (2020-03-24T00:53:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.