DP-VOXLET: Provable Speaker Anonymization for Disentangled Speech Representations

• URL: http://arxiv.org/abs/2608.30969v1
• Date: Mon, 31 Aug 2026 15:34:16 GMT
• Title: DP-VOXLET: Provable Speaker Anonymization for Disentangled Speech Representations
• Abstract summary: Systems for speaker anonymization obfuscate the speaker of an utterance, while maintaining its original semantic contents and prosody.<n>Recent solutions for speaker anonymization rely on learned representations that disentangle an utterance into semantic contents and speaker properties.<n>We introduce speaker differential privacy, a formal definition of speaker anonymization based on the framework of differential privacy, and a mechanism for speaker anonymization that provably satisfies the definition.
• Score: 2.565220169948739
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Systems for speaker anonymization obfuscate the speaker of an utterance, while maintaining its original semantic contents and prosody. Recent solutions for speaker anonymization rely on learned representations that disentangle an utterance into semantic contents and speaker properties. To anonymize an utterance, these systems replace the speaker properties while leaving the semantic contents unchanged---an approach that can produce strong results on empirical measures of privacy. In this work, we introduce speaker differential privacy, a formal definition of speaker anonymization based on the framework of differential privacy, and a mechanism for speaker anonymization that provably satisfies the definition. In contrast to prior heuristic-based anonymization systems, our approach enables a provable lower bound on re-identification success rate (e.g. equal error rate) for any possible adversary. We implement our approach in a framework that is compatible with existing disentangled representations. Compared to the prior work on differential privacy for speaker anonymization, our approach achieves significantly higher utility.

Related papers

• A Large-Scale Per-Speaker Analysis of Re-identification Risk in Speech Anonymization [14.038986003108201]
  We conduct a large-scale per-speaker privacy analysis using a linkability-based metric under a worst-case scenario.<n>Nearly 5,000 speakers are evaluated across multiple anonymization systems, attacker architectures, and conversation lengths.
  arXiv  Detail & Related papers  (2026-06-05T12:21:25Z)
• Content Anonymization for Privacy in Long-form Audio [9.679458545535388]
  Long-form audio is commonplace in domains such as interviews, phone calls, and meetings.<n> given multiple utterances from the same speaker, an attacker could exploit an individual's vocabulary, syntax, and turns of phrase.<n>We propose new content anonymization approaches to address this risk.
  arXiv  Detail & Related papers  (2025-10-14T17:52:50Z)
• VoxGuard: Evaluating User and Attribute Privacy in Speech via Membership Inference Attacks [51.68795949691009]
  We introduce VoxGuard, a framework grounded in differential privacy and membership inference.<n>For attributes, we show that simple transparent attacks recover gender and accent with near-perfect accuracy even after anonymization.<n>Our results demonstrate that EER substantially underestimates leakage, highlighting the need for low-FPR evaluation.
  arXiv  Detail & Related papers  (2025-09-22T20:57:48Z)
• A Benchmark for Multi-speaker Anonymization [9.990701310620368]
  We present an attempt to provide a multi-speaker anonymization benchmark.<n>We also discuss the privacy leakage of overlapping conversations.<n>Experiments conducted on both non-overlap simulated and real-world datasets demonstrate the effectiveness of the multi-speaker anonymization system.
  arXiv  Detail & Related papers  (2024-07-08T04:48:43Z)
• Improving Speaker Diarization using Semantic Information: Joint Pairwise Constraints Propagation [53.01238689626378]
  We propose a novel approach to leverage semantic information in speaker diarization systems. We introduce spoken language understanding modules to extract speaker-related semantic information. We present a novel framework to integrate these constraints into the speaker diarization pipeline.
  arXiv  Detail & Related papers  (2023-09-19T09:13:30Z)
• Anonymizing Speech: Evaluating and Designing Speaker Anonymization Techniques [1.2691047660244337]
  The growing use of voice user interfaces has led to a surge in the collection and storage of speech data. This thesis proposes solutions for anonymizing speech and evaluating the degree of the anonymization.
  arXiv  Detail & Related papers  (2023-08-05T16:14:17Z)
• Differentially Private Speaker Anonymization [44.90119821614047]
  Sharing real-world speech utterances is key to the training and deployment of voice-based services. Speaker anonymization aims to remove speaker information from a speech utterance while leaving its linguistic and prosodic attributes intact. We show that disentanglement is indeed not perfect: linguistic and prosodic attributes still contain speaker information.
  arXiv  Detail & Related papers  (2022-02-23T23:20:30Z)
• Design Choices for X-vector Based Speaker Anonymization [48.46018902334472]
  We present a flexible pseudo-speaker selection technique as a baseline for the first VoicePrivacy Challenge. Experiments are performed using datasets derived from LibriSpeech to find the optimal combination of design choices in terms of privacy and utility.
  arXiv  Detail & Related papers  (2020-05-18T11:32:14Z)
• Speaker Diarization with Lexical Information [59.983797884955]
  This work presents a novel approach for speaker diarization to leverage lexical information provided by automatic speech recognition. We propose a speaker diarization system that can incorporate word-level speaker turn probabilities with speaker embeddings into a speaker clustering process to improve the overall diarization accuracy.
  arXiv  Detail & Related papers  (2020-04-13T17:16:56Z)
• Speech Enhancement using Self-Adaptation and Multi-Head Self-Attention [70.82604384963679]
  This paper investigates a self-adaptation method for speech enhancement using auxiliary speaker-aware features. We extract a speaker representation used for adaptation directly from the test utterance.
  arXiv  Detail & Related papers  (2020-02-14T05:05:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.