A Large-Scale Per-Speaker Analysis of Re-identification Risk in Speech Anonymization

• URL: http://arxiv.org/abs/2606.07210v1
• Date: Fri, 05 Jun 2026 12:21:25 GMT
• Title: A Large-Scale Per-Speaker Analysis of Re-identification Risk in Speech Anonymization
• Abstract summary: We conduct a large-scale per-speaker privacy analysis using a linkability-based metric under a worst-case scenario.<n>Nearly 5,000 speakers are evaluated across multiple anonymization systems, attacker architectures, and conversation lengths.
• Score: 14.038986003108201
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Speech anonymization is commonly evaluated using averagecase metrics such as the equal error rate, which can hide large disparities in re-identification risks across individuals. In this paper, we conduct a large-scale per-speaker privacy analysis using a linkability-based metric under a worst-case scenario. Nearly 5,000 speakers are evaluated across multiple anonymization systems, attacker architectures, and conversation lengths. While linkability scores are highly polarized at the speaker level, the sets of easy to re-identify and hard to re-identify speakers vary substantially across configurations. We show that no single factor explains speaker vulnerability. Instead, the re-identification risk emerges from the interaction between the attacker, the anonymizer, and the amount of available speech. These results challenge the notion of intrinsic speaker-level privacy risks and emphasize the need for evaluation protocols that are explicitly conditioned on the attacker and anonymizer.

Related papers

• DP-VOXLET: Provable Speaker Anonymization for Disentangled Speech Representations [2.565220169948739]
  Systems for speaker anonymization obfuscate the speaker of an utterance, while maintaining its original semantic contents and prosody.<n>Recent solutions for speaker anonymization rely on learned representations that disentangle an utterance into semantic contents and speaker properties.<n>We introduce speaker differential privacy, a formal definition of speaker anonymization based on the framework of differential privacy, and a mechanism for speaker anonymization that provably satisfies the definition.
  arXiv  Detail & Related papers  (2026-08-31T15:34:16Z)
• Boundary-targeted Membership Inference Attacks on Safety Classifiers [70.20833439671131]
  Safety classifiers are trained on sensitive datasets including discussions of self-harm and mental health.<n>We introduce a new boundary-targeted selection strategy that identifies low confidence examples.<n>Experiments show that an adversary can recover 19% of the conversations a safety classifier flagged as indicating user distress, at a 5% false-positive rate.<n>This is $3.5$ times more than attacking using state-of-the-art MIA methods alone.
  arXiv  Detail & Related papers  (2026-05-21T12:05:22Z)
• Voice Privacy from an Attribute-based Perspective [0.10195618602298682]
  We introduce an attribute-based perspective, where we measure privacy protection in terms of comparisons between sets of speaker attributes.<n>We analyze privacy impact by calculating speaker uniqueness for ground truth attributes, attributes inferred on the original speech, and attributes inferred on speech protected with standard anonymization.
  arXiv  Detail & Related papers  (2026-03-19T09:53:30Z)
• VoxGuard: Evaluating User and Attribute Privacy in Speech via Membership Inference Attacks [51.68795949691009]
  We introduce VoxGuard, a framework grounded in differential privacy and membership inference.<n>For attributes, we show that simple transparent attacks recover gender and accent with near-perfect accuracy even after anonymization.<n>Our results demonstrate that EER substantially underestimates leakage, highlighting the need for low-FPR evaluation.
  arXiv  Detail & Related papers  (2025-09-22T20:57:48Z)
• Evaluating Identity Leakage in Speaker De-Identification Systems [1.7699344561127388]
  Speaker de-identification aims to conceal a speaker's identity while preserving intelligibility of the underlying speech.<n>We introduce a benchmark that quantifies residual identity leakage with three complementary error rates.<n> Evaluation results reveal that all state-of-the-art speaker de-identification systems leak identity information.
  arXiv  Detail & Related papers  (2025-08-19T17:20:25Z)
• Towards Low-Latency Tracking of Multiple Speakers With Short-Context Speaker Embeddings [52.985061676464554]
  We propose a Knowledge Distillation based training approach for short context speaker embedding extraction.<n>We leverage the spatial information of the speaker of interest using beamforming to reduce overlap.<n>Results demonstrate that our models are effective at short-context embedding extraction and more robust to overlap.
  arXiv  Detail & Related papers  (2025-08-18T11:32:13Z)
• A Benchmark for Multi-speaker Anonymization [9.990701310620368]
  We present an attempt to provide a multi-speaker anonymization benchmark.<n>We also discuss the privacy leakage of overlapping conversations.<n>Experiments conducted on both non-overlap simulated and real-world datasets demonstrate the effectiveness of the multi-speaker anonymization system.
  arXiv  Detail & Related papers  (2024-07-08T04:48:43Z)
• Towards Probing Speech-Specific Risks in Large Multimodal Models: A Taxonomy, Benchmark, and Insights [50.89022445197919]
  We propose a speech-specific risk taxonomy, covering 8 risk categories under hostility (malicious sarcasm and threats), malicious imitation (age, gender, ethnicity), and stereotypical biases (age, gender, ethnicity) Based on the taxonomy, we create a small-scale dataset for evaluating current LMMs capability in detecting these categories of risk.
  arXiv  Detail & Related papers  (2024-06-25T10:08:45Z)
• Anonymizing Speech: Evaluating and Designing Speaker Anonymization Techniques [1.2691047660244337]
  The growing use of voice user interfaces has led to a surge in the collection and storage of speech data. This thesis proposes solutions for anonymizing speech and evaluating the degree of the anonymization.
  arXiv  Detail & Related papers  (2023-08-05T16:14:17Z)
• Speaker De-identification System using Autoencoders and Adversarial Training [58.720142291102135]
  We propose a speaker de-identification system based on adversarial training and autoencoders. Experimental results show that combining adversarial learning and autoencoders increase the equal error rate of a speaker verification system.
  arXiv  Detail & Related papers  (2020-11-09T19:22:05Z)
• Design Choices for X-vector Based Speaker Anonymization [48.46018902334472]
  We present a flexible pseudo-speaker selection technique as a baseline for the first VoicePrivacy Challenge. Experiments are performed using datasets derived from LibriSpeech to find the optimal combination of design choices in terms of privacy and utility.
  arXiv  Detail & Related papers  (2020-05-18T11:32:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.