Differentially Private Subspace Fine-Tuning for Large Language Models

• URL: http://arxiv.org/abs/2601.11113v1
• Date: Fri, 16 Jan 2026 09:15:46 GMT
• Title: Differentially Private Subspace Fine-Tuning for Large Language Models
• Authors: Lele Zheng, Xiang Wang, Tao Zhang, Yang Cao, Ke Cheng, Yulong Shen,
• Abstract summary: Differential privacy (DP) offers rigorous privacy guarantees and has been widely adopted in fine-tuning.<n>We propose DP-SFT, a two-stage subspace fine-tuning method that substantially reduces noise magnitude while preserving formal DP guarantees.<n> Experiments on multiple datasets demonstrate that DP-SFT enhances accuracy and stability under rigorous DP constraints, accelerates convergence, and substantial gains over DP fine-tuning baselines.
• Score: 23.87587995037086
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fine-tuning large language models on downstream tasks is crucial for realizing their cross-domain potential but often relies on sensitive data, raising privacy concerns. Differential privacy (DP) offers rigorous privacy guarantees and has been widely adopted in fine-tuning; however, naively injecting noise across the high-dimensional parameter space creates perturbations with large norms, degrading performance and destabilizing training. To address this issue, we propose DP-SFT, a two-stage subspace fine-tuning method that substantially reduces noise magnitude while preserving formal DP guarantees. Our intuition is that, during fine-tuning, significant parameter updates lie within a low-dimensional, task-specific subspace, while other directions change minimally. Hence, we only inject DP noise into this subspace to protect privacy without perturbing irrelevant parameters. In phase one, we identify the subspace by analyzing principal gradient directions to capture task-specific update signals. In phase two, we project full gradients onto this subspace, add DP noise, and map the perturbed gradients back to the original parameter space for model updates, markedly lowering noise impact. Experiments on multiple datasets demonstrate that DP-SFT enhances accuracy and stability under rigorous DP constraints, accelerates convergence, and achieves substantial gains over DP fine-tuning baselines.

Related papers

• Enhancing DPSGD via Per-Sample Momentum and Low-Pass Filtering [4.9871580445771455]
  Differentially Private Gradient Descent (DPSGD) is widely used to train deep neural networks with formal privacy guarantees.<n>Existing techniques typically address only one of these issues, as reducing DP noise can exacerbate clipping bias and vice-versa.<n>We propose a novel method, emphDP-PMLF, which integrates per-sample momentum with a low-pass filtering strategy to simultaneously mitigate DP noise and clipping bias.
  arXiv  Detail & Related papers  (2025-11-11T23:40:02Z)
• Adaptive Token-Weighted Differential Privacy for LLMs: Not All Tokens Require Equal Protection [12.047350336564193]
  We operationalize this insight through Adaptive Token-Weighted Differential Privacy (ATDP)<n>ATDP adaptively assigns different gradient weights to sensitive and non-sensitive tokens.<n>It can be seamlessly integrated into any existing DP-based fine-tuning pipeline.
  arXiv  Detail & Related papers  (2025-09-27T10:51:07Z)
• Technical Report: Full Version of Analyzing and Optimizing Perturbation of DP-SGD Geometrically [7.905629859216635]
  We first generalize DP-SGD and theoretically derive the impact of DP noise on the training process.<n>Our analysis reveals that, in terms of a perturbed gradient, only the noise on direction has eminent impact on the model efficiency.<n>We design a geometric strategy GeoDP within the DP framework, which perturbs the direction and the magnitude of a gradient.
  arXiv  Detail & Related papers  (2025-04-08T02:26:10Z)
• DOPPLER: Differentially Private Optimizers with Low-pass Filter for Privacy Noise Reduction [47.65999101635902]
  Differentially private (DP) training prevents the leakage of sensitive information in the collected training data from trained machine learning models. We develop a new component, called DOPPLER, which works by effectively amplifying the gradient while DP noise within this frequency domain. Our experiments show that the proposed DPs with a lowpass filter outperform their counterparts without the filter by 3%-10% in test accuracy.
  arXiv  Detail & Related papers  (2024-08-24T04:27:07Z)
• Stable Neighbor Denoising for Source-free Domain Adaptive Segmentation [91.83820250747935]
  Pseudo-label noise is mainly contained in unstable samples in which predictions of most pixels undergo significant variations during self-training. We introduce the Stable Neighbor Denoising (SND) approach, which effectively discovers highly correlated stable and unstable samples. SND consistently outperforms state-of-the-art methods in various SFUDA semantic segmentation settings.
  arXiv  Detail & Related papers  (2024-06-10T21:44:52Z)
• Towards the Flatter Landscape and Better Generalization in Federated Learning under Client-level Differential Privacy [67.33715954653098]
  We propose a novel DPFL algorithm named DP-FedSAM, which leverages gradient perturbation to mitigate the negative impact of DP. Specifically, DP-FedSAM integrates Sharpness Aware of Minimization (SAM) to generate local flatness models with stability and weight robustness. To further reduce the magnitude random noise while achieving better performance, we propose DP-FedSAM-$top_k$ by adopting the local update sparsification technique.
  arXiv  Detail & Related papers  (2023-05-01T15:19:09Z)
• Make Landscape Flatter in Differentially Private Federated Learning [69.78485792860333]
  We propose a novel DPFL algorithm named DP-FedSAM, which leverages gradient perturbation to mitigate the negative impact of DP. Specifically, DP-FedSAM integrates local flatness models with better stability and weight robustness, which results in the small norm of local updates and robustness to DP noise. Our algorithm achieves state-of-the-art (SOTA) performance compared with existing SOTA baselines in DPFL.
  arXiv  Detail & Related papers  (2023-03-20T16:27:36Z)
• Dynamic Differential-Privacy Preserving SGD [19.273542515320372]
  Differentially-Private Gradient Descent (DP-SGD) prevents training-data privacy breaches by adding noise to the clipped gradient during SGD training. The same clipping operation and additive noise across training steps results in unstable updates and even a ramp-up period. We propose the dynamic DP-SGD, which has a lower privacy cost than the DP-SGD during updates until they achieve the same target privacy budget.
  arXiv  Detail & Related papers  (2021-10-30T04:45:11Z)
• Smoothed Differential Privacy [55.415581832037084]
  Differential privacy (DP) is a widely-accepted and widely-applied notion of privacy based on worst-case analysis. In this paper, we propose a natural extension of DP following the worst average-case idea behind the celebrated smoothed analysis. We prove that any discrete mechanism with sampling procedures is more private than what DP predicts, while many continuous mechanisms with sampling procedures are still non-private under smoothed DP.
  arXiv  Detail & Related papers  (2021-07-04T06:55:45Z)
• Bypassing the Ambient Dimension: Private SGD with Gradient Subspace Identification [47.23063195722975]
  Differentially private SGD (DP-SGD) is one of the most popular methods for solving differentially private empirical risk minimization (ERM) Due to its noisy perturbation on each gradient update, the error rate of DP-SGD scales with the ambient dimension $p$, the number of parameters in the model. We propose Projected DP-SGD that performs noise reduction by projecting the noisy gradients to a low-dimensional subspace.
  arXiv  Detail & Related papers  (2020-07-07T22:31:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.