Tuning without Peeking: Provable Privacy and Generalization Bounds for LLM Post-Training

• URL: http://arxiv.org/abs/2507.01752v2
• Date: Fri, 10 Oct 2025 09:08:31 GMT
• Title: Tuning without Peeking: Provable Privacy and Generalization Bounds for LLM Post-Training
• Authors: Ismail Labiad, Mathurin Videau, Matthieu Kowalski, Marc Schoenauer, Alessandro Leite, Julia Kempe, Olivier Teytaud,
• Abstract summary: BBoxER induces an information bottleneck via implicit compression of training data.<n>We provide non-vacuous generalization bounds and strong theoretical guarantees for differential privacy, to data poisoning attacks, and extraction attacks.
• Score: 49.75298684433045
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Gradient-based optimization is the workhorse of deep learning, offering efficient and scalable training via backpropagation. However, exposing gradients during training can leak sensitive information about the underlying data, raising privacy and security concerns such as susceptibility to data poisoning attacks. In contrast, black box optimization methods, which treat the model as an opaque function, relying solely on function evaluations to guide optimization, offer a promising alternative in scenarios where data access is restricted, adversarial risks are high, or overfitting is a concern. This paper introduces BBoxER, an evolutionary black-box method for LLM post-training that induces an information bottleneck via implicit compression of the training data. Leveraging the tractability of information flow, we provide non-vacuous generalization bounds and strong theoretical guarantees for differential privacy, robustness to data poisoning attacks, and extraction attacks. In experiments with LLMs, we demonstrate empirically that black-box optimization methods-despite the scalability and computational challenges inherent to black-box approaches-are able to learn, showing how a few iterations of BBoxER improve performance, generalize well on a benchmark of reasoning datasets, and are robust to membership inference attacks. This positions BBoxER as an attractive add-on on top of gradient-based optimization, offering suitability for deployment in restricted or privacy-sensitive environments while also providing non-vacuous generalization guarantees.

Related papers

• Training Data Selection with Gradient Orthogonality for Efficient Domain Adaptation [21.694351921779845]
  Fine-tuning large language models for specialized domains often necessitates a trade-off between acquiring domain expertise and retaining general reasoning capabilities.<n>We propose Orthogonal Gradient Selection (OGS), a data-centric method that harmonizes domain performance, general capability retention, and training efficiency.
  arXiv  Detail & Related papers  (2026-02-06T03:41:40Z)
• CATNIP: LLM Unlearning via Calibrated and Tokenized Negative Preference Alignment [14.853204323785334]
  Existing approaches, rooted in Gradient Ascent (GA), often degrade general domain knowledge while relying on retention data or curated contrastive pairs.<n>We develop a principled method that rescales unlearning effects in proportion to the model's token-level confidence.<n>Our work enables effective unlearning without requiring retention data or contrastive unlearning response pairs.
  arXiv  Detail & Related papers  (2026-02-02T21:23:54Z)
• IOTA: Corrective Knowledge-Guided Prompt Learning via Black-White Box Framework [57.66924056568018]
  We propose a novel black-whIte bOx prompT leArning framework (IOTA) for adapting pre-trained models to downstream tasks.<n>IOTA integrates a data-driven Black Box module with a knowledge-driven White Box module for downstream task adaptation.
  arXiv  Detail & Related papers  (2026-01-28T12:03:48Z)
• Black-Box Membership Inference Attack for LVLMs via Prior Knowledge-Calibrated Memory Probing [25.68362027128315]
  Large vision-language models (LVLMs) derive their capabilities from extensive training on vast corpora of visual and textual data.<n>We propose the first black-box MIA framework for LVLMs, based on a prior knowledge-calibrated memory probing mechanism.<n>Our method effectively identifies training data of LVLMs in a purely black-box setting and even achieves performance comparable to gray-box and white-box methods.
  arXiv  Detail & Related papers  (2025-11-03T13:16:30Z)
• Privacy-Utility Trade-off in Data Publication: A Bilevel Optimization Framework with Curvature-Guided Perturbation [22.727580097886747]
  We introduce a novel bilevel optimization framework for the publication of private datasets.<n>In the upper-level task, a discriminator guides the generation process to ensure that latent variables are mapped to high-quality samples.<n>In the lower-level task, our framework employs local extrinsic curvature on the data manifold as a quantitative measure of individual vulnerability to MIA.
  arXiv  Detail & Related papers  (2025-09-02T07:44:21Z)
• SOFT: Selective Data Obfuscation for Protecting LLM Fine-tuning against Membership Inference Attacks [17.77094760401298]
  We study the vulnerability of fine-tuned large language models to membership inference attacks (MIAs)<n>We propose SOFT, a novel defense technique that mitigates privacy leakage by leveraging influential data selection with an adjustable parameter to balance utility preservation and privacy protection.
  arXiv  Detail & Related papers  (2025-06-12T07:23:56Z)
• When Better Features Mean Greater Risks: The Performance-Privacy Trade-Off in Contrastive Learning [9.660010886245155]
  This paper systematically investigates the privacy threats posed by membership inference attacks (MIAs) targeting encoder models.<n>We propose a novel membership inference attack method based on the p-norm of feature vectors, termed the Embedding Lp-Norm Likelihood Attack (LpLA)
  arXiv  Detail & Related papers  (2025-06-06T05:03:29Z)
• ESLM: Risk-Averse Selective Language Modeling for Efficient Pretraining [53.893792844055106]
  Large language model pretraining is compute-intensive, yet many tokens contribute marginally to learning, resulting in inefficiency.<n>We introduce Selective Efficient Language Modeling, a risk-aware algorithm that improves training efficiency and distributional robustness by performing online token-level batch selection.<n> Experiments on GPT-2 pretraining show that ESLM significantly reduces training FLOPs while maintaining or improving both perplexity and downstream performance compared to baselines.
  arXiv  Detail & Related papers  (2025-05-26T12:23:26Z)
• Efficient and Private: Memorisation under differentially private parameter-efficient fine-tuning in language models [2.3281513013731145]
  Fine-tuning large language models (LLMs) for specific tasks introduces privacy risks, as models may inadvertently memorise and leak sensitive training data. Differential Privacy (DP) offers a solution to mitigate these risks, but introduces significant computational and performance trade-offs. We show that PEFT methods achieve comparable performance to standard fine-tuning while requiring fewer parameters and significantly reducing privacy leakage.
  arXiv  Detail & Related papers  (2024-11-24T13:17:36Z)
• Pseudo-Probability Unlearning: Towards Efficient and Privacy-Preserving Machine Unlearning [59.29849532966454]
  We propose PseudoProbability Unlearning (PPU), a novel method that enables models to forget data to adhere to privacy-preserving manner. Our method achieves over 20% improvements in forgetting error compared to the state-of-the-art.
  arXiv  Detail & Related papers  (2024-11-04T21:27:06Z)
• FedDTPT: Federated Discrete and Transferable Prompt Tuning for Black-Box Large Language Models [14.719919025265224]
  Fine-tuning large language models (LLMs) with data from specific scenarios poses privacy leakage risks. We propose for the first time a federated discrete and transferable prompt tuning, namely FedDTPT, for black-box large language models. Our approach achieves higher accuracy, reduced communication overhead, and robustness to non-iid data in a black-box setting.
  arXiv  Detail & Related papers  (2024-11-01T19:19:23Z)
• Protecting Privacy Through Approximating Optimal Parameters for Sequence Unlearning in Language Models [37.172662930947446]
  Language models (LMs) are potentially vulnerable to extraction attacks, which represent a significant privacy risk. We propose Privacy Protection via Optimal Parameters (POP), a novel unlearning method that effectively forgets the target token sequences from the pretrained LM. POP exhibits remarkable retention performance post-unlearning across 9 classification and 4 dialogue benchmarks, outperforming the state-of-the-art by a large margin.
  arXiv  Detail & Related papers  (2024-06-20T08:12:49Z)
• RigorLLM: Resilient Guardrails for Large Language Models against Undesired Content [62.685566387625975]
  Current mitigation strategies, while effective, are not resilient under adversarial attacks. This paper introduces Resilient Guardrails for Large Language Models (RigorLLM), a novel framework designed to efficiently moderate harmful and unsafe inputs.
  arXiv  Detail & Related papers  (2024-03-19T07:25:02Z)
• Do Gradient Inversion Attacks Make Federated Learning Unsafe? [70.0231254112197]
  Federated learning (FL) allows the collaborative training of AI models without needing to share raw data. Recent works on the inversion of deep neural networks from model gradients raised concerns about the security of FL in preventing the leakage of training data. In this work, we show that these attacks presented in the literature are impractical in real FL use-cases and provide a new baseline attack.
  arXiv  Detail & Related papers  (2022-02-14T18:33:12Z)
• Virtual Data Augmentation: A Robust and General Framework for Fine-tuning Pre-trained Models [51.46732511844122]
  Powerful pre-trained language models (PLM) can be fooled by small perturbations or intentional attacks. We present Virtual Data Augmentation (VDA), a general framework for robustly fine-tuning PLMs. Our approach is able to improve the robustness of PLMs and alleviate the performance degradation under adversarial attacks.
  arXiv  Detail & Related papers  (2021-09-13T09:15:28Z)
• Boosting Weakly Supervised Object Detection via Learning Bounding Box Adjusters [76.36104006511684]
  Weakly-supervised object detection (WSOD) has emerged as an inspiring recent topic to avoid expensive instance-level object annotations. We defend the problem setting for improving localization performance by leveraging the bounding box regression knowledge from a well-annotated auxiliary dataset. Our method performs favorably against state-of-the-art WSOD methods and knowledge transfer model with similar problem setting.
  arXiv  Detail & Related papers  (2021-08-03T13:38:20Z)
• FedBoosting: Federated Learning with Gradient Protected Boosting for Text Recognition [7.988454173034258]
  Federated Learning (FL) framework allows learning a shared model collaboratively without data being centralized or shared among data owners. We show in this paper that the generalization ability of the joint model is poor on Non-Independent and Non-Identically Distributed (Non-IID) data. We propose a novel boosting algorithm for FL to address both the generalization and gradient leakage issues.
  arXiv  Detail & Related papers  (2020-07-14T18:47:23Z)
• Chance-Constrained Trajectory Optimization for Safe Exploration and Learning of Nonlinear Systems [81.7983463275447]
  Learning-based control algorithms require data collection with abundant supervision for training. We present a new approach for optimal motion planning with safe exploration that integrates chance-constrained optimal control with dynamics learning and feedback control.
  arXiv  Detail & Related papers  (2020-05-09T05:57:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.