Adaptive Backtracking for Privacy Protection in Large Language Models

• URL: http://arxiv.org/abs/2508.06087v1
• Date: Fri, 08 Aug 2025 07:29:33 GMT
• Title: Adaptive Backtracking for Privacy Protection in Large Language Models
• Abstract summary: This paper introduces a novel objective: enterprise-oriented privacy concerns.<n>Existing methods such as data sanitization severely degrade model performance.<n>We propose ABack, a training-free mechanism that leverages a Hidden State Model.
• Score: 22.340767807000212
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The preservation of privacy has emerged as a critical topic in the era of artificial intelligence. However, current work focuses on user-oriented privacy, overlooking severe enterprise data leakage risks exacerbated by the Retrieval-Augmented Generation paradigm. To address this gap, our paper introduces a novel objective: enterprise-oriented privacy concerns. Achieving this objective requires overcoming two fundamental challenges: existing methods such as data sanitization severely degrade model performance, and the field lacks public datasets for evaluation. We address these challenges with several solutions. (1) To prevent performance degradation, we propose ABack, a training-free mechanism that leverages a Hidden State Model to pinpoint the origin of a leakage intention and rewrite the output safely. (2) To solve the lack of datasets, we construct PriGenQA, a new benchmark for enterprise privacy scenarios in healthcare and finance. To ensure a rigorous evaluation, we move beyond simple static attacks by developing a powerful adaptive attacker with Group Relative Policy Optimization. Experiments show that against this superior adversary, ABack improves the overall privacy utility score by up to 15\% over strong baselines, avoiding the performance trade-offs of prior methods.

Related papers

• A Dual Perspective on Synthetic Trajectory Generators: Utility Framework and Privacy Vulnerabilities [0.43165068185800287]
  This paper introduces a new framework for utility evaluation.<n>We provide evidence that privacy evaluation remains a great challenge to consider.<n>We propose a new membership inference attack against a subcategory of generative models.
  arXiv  Detail & Related papers  (2026-04-21T16:42:33Z)
• Reinforcement-Guided Synthetic Data Generation for Privacy-Sensitive Identity Recognition [60.52810518437911]
  High-fidelity generative models are increasingly needed in privacy-sensitive scenarios.<n> limited data leads to poor generative models, which in turn fail to mitigate data scarcity.<n>We propose a reinforcement-guided synthetic data generation framework that adapts general-domain generative priors to identity recognition tasks.
  arXiv  Detail & Related papers  (2026-04-09T06:52:03Z)
• Neural Gate: Mitigating Privacy Risks in LVLMs via Neuron-Level Gradient Gating [71.55435880263238]
  We introduce Neural Gate, a novel method for mitigating privacy risks through neuron-level model editing.<n>Our method improves a model's privacy safeguards by increasing its rate of refusal for privacy-related questions.
  arXiv  Detail & Related papers  (2026-03-13T03:03:20Z)
• Towards Benchmarking Privacy Vulnerabilities in Selective Forgetting with Large Language Models [28.389198065125314]
  selective forgetting (also known as machine unlearning) has shown promise for privacy and data removal tasks.<n>Despite its promise, selective forgetting raises significant privacy concerns.<n>We present the first comprehensive benchmark for evaluating privacy vulnerabilities in selective forgetting.
  arXiv  Detail & Related papers  (2025-12-19T20:04:06Z)
• Privacy-Utility Trade-off in Data Publication: A Bilevel Optimization Framework with Curvature-Guided Perturbation [22.727580097886747]
  We introduce a novel bilevel optimization framework for the publication of private datasets.<n>In the upper-level task, a discriminator guides the generation process to ensure that latent variables are mapped to high-quality samples.<n>In the lower-level task, our framework employs local extrinsic curvature on the data manifold as a quantitative measure of individual vulnerability to MIA.
  arXiv  Detail & Related papers  (2025-09-02T07:44:21Z)
• Privacy-Preserving Federated Embedding Learning for Localized Retrieval-Augmented Generation [60.81109086640437]
  We propose a novel framework called Federated Retrieval-Augmented Generation (FedE4RAG)<n>FedE4RAG facilitates collaborative training of client-side RAG retrieval models.<n>We apply homomorphic encryption within federated learning to safeguard model parameters.
  arXiv  Detail & Related papers  (2025-04-27T04:26:02Z)
• Personalized Language Models via Privacy-Preserving Evolutionary Model Merging [53.97323896430374]
  Personalization in language models aims to tailor model behavior to individual users or user groups.<n>We propose Privacy-Preserving Model Merging via Evolutionary Algorithms (PriME)<n>PriME employs gradient-free methods to directly optimize utility while reducing privacy risks.<n>Experiments on the LaMP benchmark show that PriME consistently outperforms a range of baselines, achieving up to a 45% improvement in task performance.
  arXiv  Detail & Related papers  (2025-03-23T09:46:07Z)
• How Breakable Is Privacy: Probing and Resisting Model Inversion Attacks in Collaborative Inference [13.453033795109155]
  Collaborative inference improves computational efficiency for edge devices by transmitting intermediate features to cloud models.<n>There is no established criterion for assessing the difficulty of model inversion attacks (MIAs)<n>We propose the first theoretical criterion to assess MIA difficulty in CI, identifying mutual information, entropy, and effective information volume as key influencing factors.
  arXiv  Detail & Related papers  (2025-01-01T13:00:01Z)
• Fed-AugMix: Balancing Privacy and Utility via Data Augmentation [15.325493418326117]
  Gradient leakage attacks pose a significant threat to the privacy guarantees of federated learning.<n>We propose a novel data augmentation-based framework designed to achieve a favorable privacy-utility trade-off.<n>Our framework incorporates the AugMix algorithm at the client level, enabling data augmentation with controllable severity.
  arXiv  Detail & Related papers  (2024-12-18T13:05:55Z)
• New Emerged Security and Privacy of Pre-trained Model: a Survey and Outlook [54.24701201956833]
  Security and privacy issues have undermined users' confidence in pre-trained models. Current literature lacks a clear taxonomy of emerging attacks and defenses for pre-trained models. This taxonomy categorizes attacks and defenses into No-Change, Input-Change, and Model-Change approaches.
  arXiv  Detail & Related papers  (2024-11-12T10:15:33Z)
• Pseudo-Probability Unlearning: Towards Efficient and Privacy-Preserving Machine Unlearning [59.29849532966454]
  We propose PseudoProbability Unlearning (PPU), a novel method that enables models to forget data to adhere to privacy-preserving manner. Our method achieves over 20% improvements in forgetting error compared to the state-of-the-art.
  arXiv  Detail & Related papers  (2024-11-04T21:27:06Z)
• Privacy-Preserving Dynamic Assortment Selection [4.399892832075127]
  This paper presents a novel framework for privacy-preserving dynamic assortment selection using the multinomial logit (MNL) bandits model. Our approach integrates noise into user utility estimates to balance between exploration and exploitation while ensuring robust privacy protection.
  arXiv  Detail & Related papers  (2024-10-29T19:28:01Z)
• Private Optimal Inventory Policy Learning for Feature-based Newsvendor with Unknown Demand [13.594765018457904]
  This paper introduces a novel approach to estimate a privacy-preserving optimal inventory policy within the f-differential privacy framework. We develop a clipped noisy gradient descent algorithm based on convolution smoothing for optimal inventory estimation. Our numerical experiments demonstrate that the proposed new method can achieve desirable privacy protection with a marginal increase in cost.
  arXiv  Detail & Related papers  (2024-04-23T19:15:43Z)
• Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Models [112.48136829374741]
  In this paper, we unveil a new vulnerability: the privacy backdoor attack. When a victim fine-tunes a backdoored model, their training data will be leaked at a significantly higher rate than if they had fine-tuned a typical model. Our findings highlight a critical privacy concern within the machine learning community and call for a reevaluation of safety protocols in the use of open-source pre-trained models.
  arXiv  Detail & Related papers  (2024-04-01T16:50:54Z)
• Visual Privacy Auditing with Diffusion Models [47.0700328585184]
  We introduce a reconstruction attack based on diffusion models (DMs) that only assumes adversary access to real-world image priors.<n>We find that (1) real-world data priors significantly influence reconstruction success, (2) current reconstruction bounds do not model the risk posed by data priors well, and DMs can serve as auditing tools for visualizing privacy leakage.
  arXiv  Detail & Related papers  (2024-03-12T12:18:55Z)
• From Mean to Extreme: Formal Differential Privacy Bounds on the Success of Real-World Data Reconstruction Attacks [54.25638567385662]
  Differential Privacy in machine learning is often interpreted as guarantees against membership inference.<n> translating DP budgets into quantitative protection against the more damaging threat of data reconstruction remains a challenging open problem.<n>This paper bridges the critical gap by deriving the first formal privacy bounds tailored to the mechanics of demonstrated "from-scratch" attacks.
  arXiv  Detail & Related papers  (2024-02-20T09:52:30Z)
• Re-thinking Data Availablity Attacks Against Deep Neural Networks [53.64624167867274]
  In this paper, we re-examine the concept of unlearnable examples and discern that the existing robust error-minimizing noise presents an inaccurate optimization objective. We introduce a novel optimization paradigm that yields improved protection results with reduced computational time requirements.
  arXiv  Detail & Related papers  (2023-05-18T04:03:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.