Alignment Defends LLMs from Property Inference Attacks

• URL: http://arxiv.org/abs/2606.10217v1
• Date: Mon, 08 Jun 2026 22:15:20 GMT
• Title: Alignment Defends LLMs from Property Inference Attacks
• Abstract summary: We propose alignment-based defenses for mitigating property inference attacks in large language models.<n>Our approach reshapes the model's output distribution towards a target property ratio via post-training alignment, without modifying the training data.
• Score: 32.582793399987274
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) are increasingly fine-tuned on domain-specific datasets that may contain sensitive, dataset-level properties. Recent work has shown that such dataset-level information can be effectively extracted through property inference attacks, posing a confidentiality risk. Existing defenses against these attacks primarily operate by modifying the training data distribution and hence require access to the original data and retraining the model, limiting their applicability to settings where data is unavailable or models are already deployed. In this work, we propose alignment-based defenses for mitigating property inference attacks in LLMs. Our approach reshapes the model's output distribution towards a target property ratio via post-training alignment, without modifying the training data. In particular, we adapt two widely used RLHF frameworks--Direct Preference Optimization (DPO) and Group Relative Policy Optimization (GRPO)--as our defenses by constructing preference pairs and defining a specific reward function respectively. Through comprehensive experiments, we show that our alignment based defenses effectively mitigate property inference attacks while maintaining a strong utility confidentiality tradeoff.

Related papers

• Benchmarking Empirical Privacy Protection for Adaptations of Large Language Models [35.164227002323706]
  We investigate privacy risks under DP adaptations in large language models (LLMs)<n>We benchmark these risks using state-of-the-art attacks such as robust membership inference and canary data extraction.<n>We find that parameter-efficient fine-tuning methods, such as LoRA, achieve the highest empirical privacy protection for OOD data.
  arXiv  Detail & Related papers  (2026-06-08T12:21:02Z)
• Adaptive Defense against Harmful Fine-Tuning for Large Language Models via Bayesian Data Scheduler [67.24175911858312]
  Harmful fine-tuning poses critical safety risks to fine-tuning-as-a-service for large language models.<n>Bayesian Data Scheduler (BDS) is an adaptive tuning-stage defense strategy with no need for attack simulation.<n>BDS learns the posterior distribution of each data point's safety attribute, conditioned on the fine-tuning and alignment datasets.
  arXiv  Detail & Related papers  (2025-10-31T04:49:37Z)
• A Vision-Language Pre-training Model-Guided Approach for Mitigating Backdoor Attacks in Federated Learning [43.847168319564844]
  We propose an FL backdoor defense framework, named CLIP-Fed, that utilizes the zero-shot learning capabilities of vision-language pre-training models.<n>Our scheme overcomes the limitations of Non-IID imposed on defense effectiveness by integrating pre-aggregation and post-aggregation defense strategies.
  arXiv  Detail & Related papers  (2025-08-14T03:39:54Z)
• Can We Infer Confidential Properties of Training Data from LLMs? [32.582793399987274]
  We introduce PropInfer, a benchmark task for evaluating property inference in large language models (LLMs)<n>Built on the ChatDoctor dataset, our benchmark includes a range of property types and task configurations.<n>We propose two tailored attacks: a prompt-based generation attack and a shadow-model attack leveraging word frequency signals.
  arXiv  Detail & Related papers  (2025-06-12T05:42:06Z)
• MISLEADER: Defending against Model Extraction with Ensembles of Distilled Models [56.09354775405601]
  Model extraction attacks aim to replicate the functionality of a black-box model through query access.<n>Most existing defenses presume that attacker queries have out-of-distribution (OOD) samples, enabling them to detect and disrupt suspicious inputs.<n>We propose MISLEADER, a novel defense strategy that does not rely on OOD assumptions.
  arXiv  Detail & Related papers  (2025-06-03T01:37:09Z)
• Unveiling the Vulnerability of Private Fine-Tuning in Split-Based Frameworks for Large Language Models: A Bidirectionally Enhanced Attack [20.727726850786386]
  BiSR is the first data reconstruction attack designed to target both the forward and backward propagation processes of split learning (SL) We propose BiSR, the first data reconstruction attack (DRA) designed to target both the forward and backward propagation processes of SL.
  arXiv  Detail & Related papers  (2024-09-02T06:01:20Z)
• Robust Utility-Preserving Text Anonymization Based on Large Language Models [80.5266278002083]
  Anonymizing text that contains sensitive information is crucial for a wide range of applications.<n>Existing techniques face the emerging challenges of the re-identification ability of large language models.<n>We propose a framework composed of three key components: a privacy evaluator, a utility evaluator, and an optimization component.
  arXiv  Detail & Related papers  (2024-07-16T14:28:56Z)
• Self-Augmented Preference Optimization: Off-Policy Paradigms for Language Model Alignment [104.18002641195442]
  We introduce Self-Augmented Preference Optimization (SAPO), an effective and scalable training paradigm that does not require existing paired data. Building on the self-play concept, which autonomously generates negative responses, we further incorporate an off-policy learning pipeline to enhance data exploration and exploitation.
  arXiv  Detail & Related papers  (2024-05-31T14:21:04Z)
• PrivacyMind: Large Language Models Can Be Contextual Privacy Protection Learners [81.571305826793]
  We introduce Contextual Privacy Protection Language Models (PrivacyMind) Our work offers a theoretical analysis for model design and benchmarks various techniques. In particular, instruction tuning with both positive and negative examples stands out as a promising method.
  arXiv  Detail & Related papers  (2023-10-03T22:37:01Z)
• The Power and Limitation of Pretraining-Finetuning for Linear Regression under Covariate Shift [127.21287240963859]
  We investigate a transfer learning approach with pretraining on the source data and finetuning based on the target data. For a large class of linear regression instances, transfer learning with $O(N2)$ source data is as effective as supervised learning with $N$ target data.
  arXiv  Detail & Related papers  (2022-08-03T05:59:49Z)
• A Regularized Implicit Policy for Offline Reinforcement Learning [54.7427227775581]
  offline reinforcement learning enables learning from a fixed dataset, without further interactions with the environment. We propose a framework that supports learning a flexible yet well-regularized fully-implicit policy. Experiments and ablation study on the D4RL dataset validate our framework and the effectiveness of our algorithmic designs.
  arXiv  Detail & Related papers  (2022-02-19T20:22:04Z)
• Property Inference Attacks Against GANs [19.443816794076763]
  We propose the first set of training dataset property inference attacks against generative adversarial networks (GANs) A successful property inference attack can allow the adversary to gain extra knowledge of the target GAN's training dataset. We propose a general attack pipeline that can be tailored to two attack scenarios, including the full black-box setting and partial black-box setting.
  arXiv  Detail & Related papers  (2021-11-15T08:57:00Z)
• Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer [137.36099660616975]
  Unsupervised adaptation adaptation (UDA) aims to transfer knowledge from a related but different well-labeled source domain to a new unlabeled target domain. Most existing UDA methods require access to the source data, and thus are not applicable when the data are confidential and not shareable due to privacy concerns. This paper aims to tackle a realistic setting with only a classification model available trained over, instead of accessing to the source data.
  arXiv  Detail & Related papers  (2020-12-14T07:28:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.