Speech Privacy Leakage from Shared Gradients in Distributed Learning

• URL: http://arxiv.org/abs/2302.10441v1
• Date: Tue, 21 Feb 2023 04:48:29 GMT
• Title: Speech Privacy Leakage from Shared Gradients in Distributed Learning
• Authors: Zhuohang Li, Jiaxin Zhang, Jian Liu
• Abstract summary: We explore methods for recovering private speech/speaker information from the shared gradients in distributed learning settings. We demonstrate the feasibility of inferring various levels of side-channel information, including speech content and speaker identity, under the distributed learning framework.
• Score: 7.8470002970302195
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Distributed machine learning paradigms, such as federated learning, have been recently adopted in many privacy-critical applications for speech analysis. However, such frameworks are vulnerable to privacy leakage attacks from shared gradients. Despite extensive efforts in the image domain, the exploration of speech privacy leakage from gradients is quite limited. In this paper, we explore methods for recovering private speech/speaker information from the shared gradients in distributed learning settings. We conduct experiments on a keyword spotting model with two different types of speech features to quantify the amount of leaked information by measuring the similarity between the original and recovered speech signals. We further demonstrate the feasibility of inferring various levels of side-channel information, including speech content and speaker identity, under the distributed learning framework without accessing the user's data.

Related papers

• Representation Learning for Audio Privacy Preservation using Source Separation and Robust Adversarial Learning [16.1694012177079]
  We propose the integration of two commonly used approaches in privacy preservation: source separation and adversarial representation learning. The proposed system learns the latent representation of audio recordings such that it prevents differentiating between speech and non-speech recordings.
  arXiv  Detail & Related papers  (2023-08-09T13:50:00Z)
• Improving the Gap in Visual Speech Recognition Between Normal and Silent Speech Based on Metric Learning [11.50011780498048]
  This paper presents a novel metric learning approach to address the performance gap between normal and silent speech in visual speech recognition (VSR) We propose to leverage the shared literal content between normal and silent speech and present a metric learning approach based on visemes. Our evaluation demonstrates that our method improves the accuracy of silent VSR, even when limited training data is available.
  arXiv  Detail & Related papers  (2023-05-23T16:20:46Z)
• Self-supervised Fine-tuning for Improved Content Representations by Speaker-invariant Clustering [78.2927924732142]
  We propose speaker-invariant clustering (Spin) as a novel self-supervised learning method. Spin disentangles speaker information and preserves content representations with just 45 minutes of fine-tuning on a single GPU.
  arXiv  Detail & Related papers  (2023-05-18T15:59:36Z)
• Adversarial Representation Learning for Robust Privacy Preservation in Audio [11.409577482625053]
  Sound event detection systems may inadvertently reveal sensitive information about users or their surroundings. We propose a novel adversarial training method for learning representations of audio recordings. The proposed method is evaluated against a baseline approach with no privacy measures and a prior adversarial training method.
  arXiv  Detail & Related papers  (2023-04-29T08:39:55Z)
• Bootstrapping meaning through listening: Unsupervised learning of spoken sentence embeddings [4.582129557845177]
  This study tackles the unsupervised learning of semantic representations for spoken utterances. We propose WavEmbed, a sequential autoencoder that predicts hidden units from a dense representation of speech. We also propose S-HuBERT to induce meaning through knowledge distillation.
  arXiv  Detail & Related papers  (2022-10-23T21:16:09Z)
• Self-Supervised Speech Representation Learning: A Review [105.1545308184483]
  Self-supervised representation learning methods promise a single universal model that would benefit a wide variety of tasks and domains. Speech representation learning is experiencing similar progress in three main categories: generative, contrastive, and predictive methods. This review presents approaches for self-supervised speech representation learning and their connection to other research areas.
  arXiv  Detail & Related papers  (2022-05-21T16:52:57Z)
• Semantics-Preserved Distortion for Personal Privacy Protection in Information Management [65.08939490413037]
  This paper suggests a linguistically-grounded approach to distort texts while maintaining semantic integrity. We present two distinct frameworks for semantic-preserving distortion: a generative approach and a substitutive approach. We also explore privacy protection in a specific medical information management scenario, showing our method effectively limits sensitive data memorization.
  arXiv  Detail & Related papers  (2022-01-04T04:01:05Z)
• Membership Inference Attacks Against Self-supervised Speech Models [62.73937175625953]
  Self-supervised learning (SSL) on continuous speech has started gaining attention. We present the first privacy analysis on several SSL speech models using Membership Inference Attacks (MIA) under black-box access.
  arXiv  Detail & Related papers  (2021-11-09T13:00:24Z)
• CAPE: Context-Aware Private Embeddings for Private Language Learning [0.5156484100374058]
  Context-Aware Private Embeddings (CAPE) is a novel approach which preserves privacy during training of embeddings. CAPE applies calibrated noise through differential privacy, preserving the encoded semantic links while obscuring sensitive information. Experimental results demonstrate that the proposed approach reduces private information leakage better than either single intervention.
  arXiv  Detail & Related papers  (2021-08-27T14:50:12Z)
• Preliminary study on using vector quantization latent spaces for TTS/VC systems with consistent performance [55.10864476206503]
  We investigate the use of quantized vectors to model the latent linguistic embedding. By enforcing different policies over the latent spaces in the training, we are able to obtain a latent linguistic embedding. Our experiments show that the voice cloning system built with vector quantization has only a small degradation in terms of perceptive evaluations.
  arXiv  Detail & Related papers  (2021-06-25T07:51:35Z)
• Protecting gender and identity with disentangled speech representations [49.00162808063399]
  We show that protecting gender information in speech is more effective than modelling speaker-identity information. We present a novel way to encode gender information and disentangle two sensitive biometric identifiers.
  arXiv  Detail & Related papers  (2021-04-22T13:31:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.