PATE-AAE: Incorporating Adversarial Autoencoder into Private Aggregation of Teacher Ensembles for Spoken Command Classification

• URL: http://arxiv.org/abs/2104.01271v1
• Date: Fri, 2 Apr 2021 23:10:57 GMT
• Title: PATE-AAE: Incorporating Adversarial Autoencoder into Private Aggregation of Teacher Ensembles for Spoken Command Classification
• Abstract summary: We propose an adversarial autoencoder (AAE) to replace generative adversarial network (GAN) in the private aggregation of teacher ensembles (PATE) AAE architecture allows us to obtain good synthetic speechnative upon a discriminative training of latent vectors. PATE-AAE improves the average classification accuracy by +$2.11%$ and +$6.60%$, respectively, when compared with alternative privacy-preserving solutions.
• Score: 53.764020085834645
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We propose using an adversarial autoencoder (AAE) to replace generative adversarial network (GAN) in the private aggregation of teacher ensembles (PATE), a solution for ensuring differential privacy in speech applications. The AAE architecture allows us to obtain good synthetic speech leveraging upon a discriminative training of latent vectors. Such synthetic speech is used to build a privacy-preserving classifier when non-sensitive data is not sufficiently available in the public domain. This classifier follows the PATE scheme that uses an ensemble of noisy outputs to label the synthetic samples and guarantee $\varepsilon$-differential privacy (DP) on its derived classifiers. Our proposed framework thus consists of an AAE-based generator and a PATE-based classifier (PATE-AAE). Evaluated on the Google Speech Commands Dataset Version II, the proposed PATE-AAE improves the average classification accuracy by +$2.11\%$ and +$6.60\%$, respectively, when compared with alternative privacy-preserving solutions, namely PATE-GAN and DP-GAN, while maintaining a strong level of privacy target at $\varepsilon$=0.01 with a fixed $\delta$=10$^{-5}$.

Related papers

• $β$-OPSD: Deriving with Policy Optimization, Training with Self-Distillation [85.07190808882523]
  On-policy self-distillation is a promising approach to improve reasoning language models.<n>We show that $$-OPSD consistently outperforms vanilla OPSD.
  arXiv  Detail & Related papers  (2026-07-30T17:41:16Z)
• REAEDP: Entropy-Calibrated Differentially Private Data Release with Formal Guarantees and Attack-Based Evaluation [7.172756746172751]
  We propose a differential privacy framework that combines entropy-calibrated histogram release, a synthetic-data release mechanism, and attack-based evaluation.<n>On the theory side, we derive an explicit sensitivity bound for Shannon entropy, together with an extension to Rényi entropy, for adjacent histogram datasets.<n>We show that a synthetic-data mechanism satisfies a formal differential privacy guarantee under the stated parameter conditions.
  arXiv  Detail & Related papers  (2026-03-14T02:23:55Z)
• Machine Learning with Privacy for Protected Attributes [56.44253915927481]
  We refine the definition of differential privacy (DP) to create a more general and flexible framework that we call feature differential privacy (FDP)<n>Our definition is simulation-based and allows for both addition/removal and replacement variants of privacy, and can handle arbitrary separation of protected and non-protected features.<n>We apply our framework to various machine learning tasks and show that it can significantly improve the utility of DP-trained models when public features are available.
  arXiv  Detail & Related papers  (2025-06-24T17:53:28Z)
• FedSVD: Adaptive Orthogonalization for Private Federated Learning with LoRA [61.79405341803085]
  Low-Rank Adaptation (LoRA) is widely used for efficient fine-tuning of language models in federated learning (FL)<n>Low-Rank Adaptation (LoRA) is widely used for efficient fine-tuning of language models in federated learning (FL)
  arXiv  Detail & Related papers  (2025-05-19T07:32:56Z)
• Optimized Tradeoffs for Private Prediction with Majority Ensembling [59.99331405291337]
  We introduce the Data-dependent Randomized Response Majority (DaRRM) algorithm. DaRRM is parameterized by a data-dependent noise function $gamma$, and enables efficient utility optimization over the class of all private algorithms. We show that DaRRM provably enjoys a privacy gain of a factor of 2 over common baselines, with fixed utility.
  arXiv  Detail & Related papers  (2024-11-27T00:48:48Z)
• Co-training for Low Resource Scientific Natural Language Inference [65.37685198688538]
  We propose a novel co-training method that assigns weights based on the training dynamics of the classifiers to the distantly supervised labels. By assigning importance weights instead of filtering out examples based on an arbitrary threshold on the predicted confidence, we maximize the usage of automatically labeled data. The proposed method obtains an improvement of 1.5% in Macro F1 over the distant supervision baseline, and substantial improvements over several other strong SSL baselines.
  arXiv  Detail & Related papers  (2024-06-20T18:35:47Z)
• The Elusive Pursuit of Replicating PATE-GAN: Benchmarking, Auditing, Debugging [11.900523702759598]
  PATE-GAN has emerged as a popular algorithm, combining Generative Adversarial Networks (GANs) with the private training approach of PATE. We analyze and benchmark six open-source PATE-GAN implementations, including three by (a subset of) the original authors. We present an in-depth privacy evaluation, including DP auditing, showing that all implementations leak more privacy than intended and uncovering 17 privacy violations and 5 other bugs.
  arXiv  Detail & Related papers  (2024-06-20T04:25:41Z)
• Closed-Form Bounds for DP-SGD against Record-level Inference [18.85865832127335]
  We focus on the popular DP-SGD algorithm, and derive simple closed-form bounds. We obtain bounds for membership inference that match state-of-the-art techniques. We present a novel data-dependent bound against attribute inference.
  arXiv  Detail & Related papers  (2024-02-22T09:26:16Z)
• Hot PATE: Private Aggregation of Distributions for Diverse Task [25.71922346848019]
  We propose Hot PATE, a variant tailored to settings where outputs are distributions.<n>We define what it means to preserve diversity and introduce an efficient aggregation mechanism that transfers diversity to the randomized output without incurring additional privacy cost.<n> Empirically, Hot PATE achieves orders-of-magnitude improvement on in-context learning tasks.
  arXiv  Detail & Related papers  (2023-12-04T18:54:34Z)
• An Experimental Study on Private Aggregation of Teacher Ensemble Learning for End-to-End Speech Recognition [51.232523987916636]
  Differential privacy (DP) is one data protection avenue to safeguard user information used for training deep models by imposing noisy distortion on privacy data. In this work, we extend PATE learning to work with dynamic patterns, namely speech, and perform one very first experimental study on ASR to avoid acoustic data leakage.
  arXiv  Detail & Related papers  (2022-10-11T16:55:54Z)
• Individual Privacy Accounting for Differentially Private Stochastic Gradient Descent [69.14164921515949]
  We characterize privacy guarantees for individual examples when releasing models trained by DP-SGD. We find that most examples enjoy stronger privacy guarantees than the worst-case bound. This implies groups that are underserved in terms of model utility simultaneously experience weaker privacy guarantees.
  arXiv  Detail & Related papers  (2022-06-06T13:49:37Z)
• Uncertainty-Autoencoder-Based Privacy and Utility Preserving Data Type Conscious Transformation [3.7315964084413173]
  We propose an adversarial learning framework that deals with the privacy-utility tradeoff problem under two conditions. Under data-type ignorant conditions, the privacy mechanism provides a one-hot encoding of categorical features, representing exactly one class. Under data-type aware conditions, the categorical variables are represented by a collection of scores, one for each class.
  arXiv  Detail & Related papers  (2022-05-04T08:40:15Z)
• Adaptive Private-K-Selection with Adaptive K and Application to Multi-label PATE [19.344510696808282]
  We provide an end-to-end Renyi DP based-framework for differentially private top-$k$ selection. We show that our construction improves the privacy-utility trade-offs compared to the previous top-$k$ selection algorithms.
  arXiv  Detail & Related papers  (2022-03-30T07:11:01Z)
• Class-Conditional Defense GAN Against End-to-End Speech Attacks [82.21746840893658]
  We propose a novel approach against end-to-end adversarial attacks developed to fool advanced speech-to-text systems such as DeepSpeech and Lingvo. Unlike conventional defense approaches, the proposed approach does not directly employ low-level transformations such as autoencoding a given input signal. Our defense-GAN considerably outperforms conventional defense algorithms in terms of word error rate and sentence level recognition accuracy.
  arXiv  Detail & Related papers  (2020-10-22T00:02:02Z)
• Differentially Private Federated Learning with Laplacian Smoothing [72.85272874099644]
  Federated learning aims to protect data privacy by collaboratively learning a model without sharing private data among users. An adversary may still be able to infer the private training data by attacking the released model. Differential privacy provides a statistical protection against such attacks at the price of significantly degrading the accuracy or utility of the trained models.
  arXiv  Detail & Related papers  (2020-05-01T04:28:38Z)
• Decentralized Differentially Private Segmentation with PATE [19.074694101300373]
  We show how PATE can be adapted for semantic segmentation. On the Brain Tumor dataset, an Autoencoder-based PATE variant achieves a higher Dice coefficient for privacy work based on noisy Federated Averaging.
  arXiv  Detail & Related papers  (2020-04-10T00:05:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.