DP-OPD: Differentially Private On-Policy Distillation for Language Models

• URL: http://arxiv.org/abs/2604.04461v1
• Date: Mon, 06 Apr 2026 06:18:03 GMT
• Title: DP-OPD: Differentially Private On-Policy Distillation for Language Models
• Abstract summary: We propose a synthesis-free framework that enforces privacy solely through DP-SGD on the student.<n>DP-OPD instantiates this idea via emphprivate generalized knowledge distillation on continuation tokens.
• Score: 15.952882124096975
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) are increasingly adapted to proprietary and domain-specific corpora that contain sensitive information, creating a tension between formal privacy guarantees and efficient deployment through model compression. Differential privacy (DP), typically enforced via DP-SGD, provides record-level protection but often incurs substantial utility loss in autoregressive generation, where optimization noise can amplify exposure bias and compounding errors along long rollouts. Existing approaches to private distillation either apply DP-SGD to both teacher and student, worsening computation and the privacy--utility tradeoff, or rely on DP synthetic text generation from a DP-trained teacher, avoiding DP on the student at the cost of DP-optimizing a large teacher and introducing an offline generation pipeline. We propose \textbf{Differentially Private On-Policy Distillation (DP-OPD)}, a synthesis-free framework that enforces privacy solely through DP-SGD on the student while leveraging a frozen teacher to provide dense token-level targets on \emph{student-generated} trajectories. DP-OPD instantiates this idea via \emph{private generalized knowledge distillation} on continuation tokens. Under a strict privacy budget ($\varepsilon=2.0$), DP-OPD improves perplexity over DP fine-tuning and off-policy DP distillation, and outperforms synthesis-based DP distillation (Yelp: 44.15$\rightarrow$41.68; BigPatent: 32.43$\rightarrow$30.63), while substantially simplifying the training pipeline. In particular, \textbf{DP-OPD collapses private compression into a single DP student-training loop} by eliminating DP teacher training and offline synthetic text generation. Code will be released upon publication at https://github.com/khademfatemeh/dp_opd.

Related papers

• DP-SelFT: Differentially Private Selective Fine-Tuning for Large Language Models [28.62057098410814]
  We propose DP-SelFT, a framework for differentially private selective fine-tuning of large language models (LLMs)<n> DP-SelFT addresses three DP-specific challenges in parameter selection: avoiding repeated privacy cost, improving stability under noisy estimates, and selecting parameters that remain useful under clipped and noisy updates.<n> Experiments show that DP-SelFT consistently improves the privacy--utility trade-off over existing DP fine-tuning baselines under the same privacy guarantees.
  arXiv  Detail & Related papers  (2026-05-17T12:55:11Z)
• Adaptive Methods Are Preferable in High Privacy Settings: An SDE Perspective [42.70658101277954]
  Differential Privacy (DP) is becoming central to large-scale training as privacy regulations tighten.<n>We revisit how noise interacts with adaptivity in optimization through the lens of differential equations.<n>We show that DP-SGD converges at a Privacy-Utility Trade-Off of $mathcalO (1/varepsilon2)$ with speed independent of $varepsilon$, while DP-SignSGD converges at a speed linear in $varepsilon$ with speed independent of $varepsilon$.
  arXiv  Detail & Related papers  (2026-03-03T18:17:57Z)
• DP-RFT: Learning to Generate Synthetic Text via Differentially Private Reinforcement Fine-Tuning [51.35628297101575]
  Differentially private (DP) synthetic data generation plays a pivotal role in developing large language models (LLMs) on private data.<n>We introduce Differentially Private Reinforcement Fine-Tuning (DP-RFT), an online reinforcement learning algorithm for synthetic data generation with LLMs.<n>We evaluate DP-RFT for long-form and domain-specific synthetic data generation, such as news articles, meeting transcripts, and medical article abstracts.
  arXiv  Detail & Related papers  (2026-02-20T22:03:56Z)
• Machine Learning with Privacy for Protected Attributes [56.44253915927481]
  We refine the definition of differential privacy (DP) to create a more general and flexible framework that we call feature differential privacy (FDP)<n>Our definition is simulation-based and allows for both addition/removal and replacement variants of privacy, and can handle arbitrary separation of protected and non-protected features.<n>We apply our framework to various machine learning tasks and show that it can significantly improve the utility of DP-trained models when public features are available.
  arXiv  Detail & Related papers  (2025-06-24T17:53:28Z)
• DP-2Stage: Adapting Language Models as Differentially Private Tabular Data Generators [47.86275136491794]
  We propose DP-2Stage, a two-stage fine-tuning framework for differentially private data generation.<n>Our empirical results show that this approach improves performance across various settings and metrics.
  arXiv  Detail & Related papers  (2024-12-03T14:10:09Z)
• Pre-training Differentially Private Models with Limited Public Data [54.943023722114134]
  differential privacy (DP) is a prominent method to gauge the degree of security provided to the models. DP is yet not capable of protecting a substantial portion of the data used during the initial pre-training stage. We develop a novel DP continual pre-training strategy using only 10% of public data. Our strategy can achieve DP accuracy of 41.5% on ImageNet-21k, as well as non-DP accuracy of 55.7% and and 60.0% on downstream tasks Places365 and iNaturalist-2021.
  arXiv  Detail & Related papers  (2024-02-28T23:26:27Z)
• Closed-Form Bounds for DP-SGD against Record-level Inference [18.85865832127335]
  We focus on the popular DP-SGD algorithm, and derive simple closed-form bounds. We obtain bounds for membership inference that match state-of-the-art techniques. We present a novel data-dependent bound against attribute inference.
  arXiv  Detail & Related papers  (2024-02-22T09:26:16Z)
• Private Fine-tuning of Large Language Models with Zeroth-order Optimization [51.19403058739522]
  Differentially private gradient descent (DP-SGD) allows models to be trained in a privacy-preserving manner.<n>We introduce DP-ZO, a private fine-tuning framework for large language models by privatizing zeroth order optimization methods.
  arXiv  Detail & Related papers  (2024-01-09T03:53:59Z)
• Differentially Private Diffusion Models [46.46256537222917]
  We build on the recent success of diffusion models (DMs) and introduce Differentially Private Diffusion Models (DPDMs) We propose noise multiplicity, a powerful modification of DP-SGD tailored to the training of DMs. We validate our novel DPDMs on image generation benchmarks and achieve state-of-the-art performance in all experiments.
  arXiv  Detail & Related papers  (2022-10-18T15:20:47Z)
• Lifelong DP: Consistently Bounded Differential Privacy in Lifelong Machine Learning [28.68587691924582]
  We show that the process of continually learning new tasks and memorizing previous tasks introduces unknown privacy risks and challenges to bound the privacy loss. We introduce a formal definition of Lifelong DP, in which the participation of any datas in the training set of any tasks is protected. We propose a scalable and heterogeneous algorithm, called L2DP-ML, to efficiently train and continue releasing new versions of an L2M model.
  arXiv  Detail & Related papers  (2022-07-26T11:55:21Z)
• Automatic Clipping: Differentially Private Deep Learning Made Easier and Stronger [39.93710312222771]
  Per-example clipping is a key algorithmic step that enables practical differential private (DP) training for deep learning models. We propose an easy-to-use replacement, called automatic clipping, that eliminates the need to tune R for any DPs.
  arXiv  Detail & Related papers  (2022-06-14T19:49:44Z)
• Large Scale Transfer Learning for Differentially Private Image Classification [51.10365553035979]
  Differential Privacy (DP) provides a formal framework for training machine learning models with individual example level privacy. Private training using DP-SGD protects against leakage by injecting noise into individual example gradients. While this result is quite appealing, the computational cost of training large-scale models with DP-SGD is substantially higher than non-private training.
  arXiv  Detail & Related papers  (2022-05-06T01:22:20Z)
• DP-InstaHide: Provably Defusing Poisoning and Backdoor Attacks with Differentially Private Data Augmentations [54.960853673256]
  We show that strong data augmentations, such as mixup and random additive noise, nullify poison attacks while enduring only a small accuracy trade-off. A rigorous analysis of DP-InstaHide shows that mixup does indeed have privacy advantages, and that training with k-way mixup provably yields at least k times stronger DP guarantees than a naive DP mechanism.
  arXiv  Detail & Related papers  (2021-03-02T23:07:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.