Differential Privacy in Continual Learning: Which Labels to Update?

• URL: http://arxiv.org/abs/2411.04680v4
• Date: Thu, 22 May 2025 13:18:55 GMT
• Title: Differential Privacy in Continual Learning: Which Labels to Update?
• Authors: Marlon Tobaben, Talal Alrawajfeh, Marcus Klasson, Mikko Heikkilä, Arno Solin, Antti Honkela,
• Abstract summary: Continual learning conflicts with strict privacy required for sensitive training data.<n>We highlight that failing to account for privacy leakage through the set of labels a model can output can break the privacy of otherwise valid DP algorithms.
• Score: 14.721537886922864
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The goal of continual learning (CL) is to retain knowledge across tasks, but this conflicts with strict privacy required for sensitive training data that prevents storing or memorising individual samples. To address that, we combine CL and differential privacy (DP). We highlight that failing to account for privacy leakage through the set of labels a model can output can break the privacy of otherwise valid DP algorithms. This is especially relevant in CL. We show that mitigating the issue with a data-independent overly large label space can have minimal negative impact on utility when fine-tuning a pre-trained model under DP, while learning the labels with a separate DP mechanism risks losing small classes.

Related papers

• Machine Learning with Privacy for Protected Attributes [56.44253915927481]
  We refine the definition of differential privacy (DP) to create a more general and flexible framework that we call feature differential privacy (FDP)<n>Our definition is simulation-based and allows for both addition/removal and replacement variants of privacy, and can handle arbitrary separation of protected and non-protected features.<n>We apply our framework to various machine learning tasks and show that it can significantly improve the utility of DP-trained models when public features are available.
  arXiv  Detail & Related papers  (2025-06-24T17:53:28Z)
• Differentially Private Random Block Coordinate Descent [51.62669821275571]
  We propose a differentially private random coordinate descent method that selects multiple coordinates with varying probabilities in each iteration using sketch matrices. Our algorithm generalizes both DP-CD and the classical DP-SGD (Differentially Private Descent), while preserving the same utility guarantees.
  arXiv  Detail & Related papers  (2024-12-22T15:06:56Z)
• Fine-Tuning Language Models with Differential Privacy through Adaptive Noise Allocation [33.795122935686706]
  We propose ANADP, a novel algorithm that adaptively allocates additive noise based on the importance of model parameters. We demonstrate that ANADP narrows the performance gap between regular fine-tuning and traditional DP fine-tuning on a series of datasets.
  arXiv  Detail & Related papers  (2024-10-03T19:02:50Z)
• Differentially Private Active Learning: Balancing Effective Data Selection and Privacy [11.716423801223776]
  We introduce differentially private active learning (DP-AL) for standard learning settings. We demonstrate that naively integrating DP-SGD training into AL presents substantial challenges in privacy budget allocation and data utilization. Our experiments on vision and natural language processing tasks show that DP-AL can improve performance for specific datasets and model architectures.
  arXiv  Detail & Related papers  (2024-10-01T09:34:06Z)
• Mind the Privacy Unit! User-Level Differential Privacy for Language Model Fine-Tuning [62.224804688233]
  differential privacy (DP) offers a promising solution by ensuring models are 'almost indistinguishable' with or without any particular privacy unit. We study user-level DP motivated by applications where it necessary to ensure uniform privacy protection across users.
  arXiv  Detail & Related papers  (2024-06-20T13:54:32Z)
• LLM-based Privacy Data Augmentation Guided by Knowledge Distillation with a Distribution Tutor for Medical Text Classification [67.92145284679623]
  We propose a DP-based tutor that models the noised private distribution and controls samples' generation with a low privacy cost. We theoretically analyze our model's privacy protection and empirically verify our model.
  arXiv  Detail & Related papers  (2024-02-26T11:52:55Z)
• Personalized Differential Privacy for Ridge Regression [3.4751583941317166]
  We introduce our novel Personalized-DP Output Perturbation method ( PDP-OP) that enables to train Ridge regression models with individual per data point privacy levels. We provide rigorous privacy proofs for our PDP-OP as well as accuracy guarantees for the resulting model. We show that PDP-OP outperforms the personalized privacy techniques of Jorgensen et al.
  arXiv  Detail & Related papers  (2024-01-30T16:00:14Z)
• Multi-label Learning from Privacy-Label [6.403667773024114]
  We propose a novel setting named Multi-Label Learning from Privacy-Label (MLLPL) During the labeling phase, each privacy-label is randomly combined with a non-privacy label to form a Privacy-Label Unit (PLU) If any label within a PLU is positive, the unit is labeled as positive; otherwise, it is labeled negative, as shown in Figure 1.
  arXiv  Detail & Related papers  (2023-12-20T09:09:56Z)
• Towards Robust Continual Learning with Bayesian Adaptive Moment Regularization [51.34904967046097]
  Continual learning seeks to overcome the challenge of catastrophic forgetting, where a model forgets previously learnt information. We introduce a novel prior-based method that better constrains parameter growth, reducing catastrophic forgetting. Results show that BAdam achieves state-of-the-art performance for prior-based methods on challenging single-headed class-incremental experiments.
  arXiv  Detail & Related papers  (2023-09-15T17:10:51Z)
• ULDP-FL: Federated Learning with Across Silo User-Level Differential Privacy [19.017342515321918]
  Differentially Private Federated Learning (DP-FL) has garnered attention as a collaborative machine learning approach that ensures formal privacy. We present Uldp-FL, a novel FL framework designed to guarantee user-level DP in cross-silo FL where a single user's data may belong to multiple silos.
  arXiv  Detail & Related papers  (2023-08-23T15:50:51Z)
• Probing the Transition to Dataset-Level Privacy in ML Models Using an Output-Specific and Data-Resolved Privacy Profile [23.05994842923702]
  We study a privacy metric that quantifies the extent to which a model trained on a dataset using a Differential Privacy mechanism is covered" by each of the distributions resulting from training on neighboring datasets. We show that the privacy profile can be used to probe an observed transition to indistinguishability that takes place in the neighboring distributions as $epsilon$ decreases.
  arXiv  Detail & Related papers  (2023-06-27T20:39:07Z)
• Considerations on the Theory of Training Models with Differential Privacy [13.782477759025344]
  In federated learning collaborative learning takes place by a set of clients who each want to remain in control of how their local training data is used. Differential privacy is one method to limit privacy leakage.
  arXiv  Detail & Related papers  (2023-03-08T15:56:27Z)
• Tight Auditing of Differentially Private Machine Learning [77.38590306275877]
  For private machine learning, existing auditing mechanisms are tight. They only give tight estimates under implausible worst-case assumptions. We design an improved auditing scheme that yields tight privacy estimates for natural (not adversarially crafted) datasets.
  arXiv  Detail & Related papers  (2023-02-15T21:40:33Z)
• Label Inference Attack against Split Learning under Regression Setting [24.287752556622312]
  We study the leakage in the scenario of the regression model, where the private labels are continuous numbers. We propose a novel learning-based attack that integrates gradient information and extra learning regularization objectives.
  arXiv  Detail & Related papers  (2023-01-18T03:17:24Z)
• How Do Input Attributes Impact the Privacy Loss in Differential Privacy? [55.492422758737575]
  We study the connection between the per-subject norm in DP neural networks and individual privacy loss. We introduce a novel metric termed the Privacy Loss-Input Susceptibility (PLIS) which allows one to apportion the subject's privacy loss to their input attributes.
  arXiv  Detail & Related papers  (2022-11-18T11:39:03Z)
• Learning versus Refutation in Noninteractive Local Differential Privacy [133.80204506727526]
  We study two basic statistical tasks in non-interactive local differential privacy (LDP): learning and refutation. Our main result is a complete characterization of the sample complexity of PAC learning for non-interactive LDP protocols.
  arXiv  Detail & Related papers  (2022-10-26T03:19:24Z)
• An Ensemble Teacher-Student Learning Approach with Poisson Sub-sampling to Differential Privacy Preserving Speech Recognition [51.20130423303659]
  We propose an ensemble learning framework with Poisson sub-sampling to train a collection of teacher models to issue some differential privacy (DP) guarantee for training data. Through boosting under DP, a student model derived from the training data suffers little model degradation from the models trained with no privacy protection. Our proposed solution leverages upon two mechanisms, namely: (i) a privacy budget amplification via Poisson sub-sampling to train a target prediction model that requires less noise to achieve a same level of privacy budget, and (ii) a combination of the sub-sampling technique and an ensemble teacher-student learning framework.
  arXiv  Detail & Related papers  (2022-10-12T16:34:08Z)
• Cooperative Self-Training for Multi-Target Adaptive Semantic Segmentation [26.79776306494929]
  We propose a self-training strategy that employs pseudo-labels to induce cooperation among multiple domain-specific classifiers. We employ feature stylization as an efficient way to generate image views that forms an integral part of self-training.
  arXiv  Detail & Related papers  (2022-10-04T13:03:17Z)
• On the utility and protection of optimization with differential privacy and classic regularization techniques [9.413131350284083]
  We study the effectiveness of the differentially-private descent (DP-SGD) algorithm against standard optimization practices with regularization techniques. We discuss differential privacy's flaws and limits and empirically demonstrate the often superior privacy-preserving properties of dropout and l2-regularization.
  arXiv  Detail & Related papers  (2022-09-07T14:10:21Z)
• On Non-Random Missing Labels in Semi-Supervised Learning [114.62655062520425]
  Semi-Supervised Learning (SSL) is fundamentally a missing label problem. We explicitly incorporate "class" into SSL. Our method not only significantly outperforms existing baselines but also surpasses other label bias removal SSL methods.
  arXiv  Detail & Related papers  (2022-06-29T22:01:29Z)
• Individual Privacy Accounting for Differentially Private Stochastic Gradient Descent [69.14164921515949]
  We characterize privacy guarantees for individual examples when releasing models trained by DP-SGD. We find that most examples enjoy stronger privacy guarantees than the worst-case bound. This implies groups that are underserved in terms of model utility simultaneously experience weaker privacy guarantees.
  arXiv  Detail & Related papers  (2022-06-06T13:49:37Z)
• SF-PATE: Scalable, Fair, and Private Aggregation of Teacher Ensembles [50.90773979394264]
  This paper studies a model that protects the privacy of individuals' sensitive information while also allowing it to learn non-discriminatory predictors. A key characteristic of the proposed model is to enable the adoption of off-the-selves and non-private fair models to create a privacy-preserving and fair model.
  arXiv  Detail & Related papers  (2022-04-11T14:42:54Z)
• Towards Differential Relational Privacy and its use in Question Answering [109.4452196071872]
  Memorization of relation between entities in a dataset can lead to privacy issues when using a trained question answering model. We quantify this phenomenon and provide a possible definition of Differential Privacy (DPRP) We illustrate concepts in experiments with largescale models for Question Answering.
  arXiv  Detail & Related papers  (2022-03-30T22:59:24Z)
• Does Label Differential Privacy Prevent Label Inference Attacks? [26.87328379562665]
  Label differential privacy (label-DP) is a popular framework for training private ML models on datasets with public features and sensitive private labels. Despite its rigorous privacy guarantee, it has been observed that in practice label-DP does not preclude label inference attacks (LIAs)
  arXiv  Detail & Related papers  (2022-02-25T20:57:29Z)
• Semi-supervised Domain Adaptive Structure Learning [72.01544419893628]
  Semi-supervised domain adaptation (SSDA) is a challenging problem requiring methods to overcome both 1) overfitting towards poorly annotated data and 2) distribution shift across domains. We introduce an adaptive structure learning method to regularize the cooperation of SSL and DA.
  arXiv  Detail & Related papers  (2021-12-12T06:11:16Z)
• Statistical Privacy Guarantees of Machine Learning Preprocessing Techniques [1.198727138090351]
  We adapt a privacy violation detection framework based on statistical methods to measure privacy levels of machine learning pipelines. We apply the newly created framework to show that resampling techniques used when dealing with imbalanced datasets cause the resultant model to leak more privacy.
  arXiv  Detail & Related papers  (2021-09-06T14:08:47Z)
• Your Classifier can Secretly Suffice Multi-Source Domain Adaptation [72.47706604261992]
  Multi-Source Domain Adaptation (MSDA) deals with the transfer of task knowledge from multiple labeled source domains to an unlabeled target domain. We present a different perspective to MSDA wherein deep models are observed to implicitly align the domains under label supervision.
  arXiv  Detail & Related papers  (2021-03-20T12:44:13Z)
• The Influence of Dropout on Membership Inference in Differentially Private Models [0.0]
  Differentially private models seek to protect the privacy of data the model is trained on. We conduct membership inference attacks against models with and without differential privacy.
  arXiv  Detail & Related papers  (2021-03-16T12:09:51Z)
• ORDisCo: Effective and Efficient Usage of Incremental Unlabeled Data for Semi-supervised Continual Learning [52.831894583501395]
  Continual learning assumes the incoming data are fully labeled, which might not be applicable in real applications. We propose deep Online Replay with Discriminator Consistency (ORDisCo) to interdependently learn a classifier with a conditional generative adversarial network (GAN) We show ORDisCo achieves significant performance improvement on various semi-supervised learning benchmark datasets for SSCL.
  arXiv  Detail & Related papers  (2021-01-02T09:04:14Z)
• Dual-Refinement: Joint Label and Feature Refinement for Unsupervised Domain Adaptive Person Re-Identification [51.98150752331922]
  Unsupervised domain adaptive (UDA) person re-identification (re-ID) is a challenging task due to the missing of labels for the target domain data. We propose a novel approach, called Dual-Refinement, that jointly refines pseudo labels at the off-line clustering phase and features at the on-line training phase. Our method outperforms the state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2020-12-26T07:35:35Z)
• Differentially Private and Fair Deep Learning: A Lagrangian Dual Approach [54.32266555843765]
  This paper studies a model that protects the privacy of the individuals sensitive information while also allowing it to learn non-discriminatory predictors. The method relies on the notion of differential privacy and the use of Lagrangian duality to design neural networks that can accommodate fairness constraints.
  arXiv  Detail & Related papers  (2020-09-26T10:50:33Z)
• Tempered Sigmoid Activations for Deep Learning with Differential Privacy [33.574715000662316]
  We show that the choice of activation function is central to bounding the sensitivity of privacy-preserving deep learning. We achieve new state-of-the-art accuracy on MNIST, FashionMNIST, and CIFAR10 without any modification of the learning procedure fundamentals.
  arXiv  Detail & Related papers  (2020-07-28T13:19:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.