DiffAnon: Diffusion-based Prosody Control for Voice Anonymization

• URL: http://arxiv.org/abs/2604.26281v1
• Date: Wed, 29 Apr 2026 04:22:59 GMT
• Title: DiffAnon: Diffusion-based Prosody Control for Voice Anonymization
• Abstract summary: We propose DiffAnon, a diffusion-based anonymization method with continuous inference-time control over prosody preservation.<n>Experiments demonstrate structured trade-off behavior, achieving strong utility while maintaining competitive privacy across controllable operating points.
• Score: 22.52514120166268
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: To preserve or not to preserve prosody is a central question in voice anonymization. Prosody conveys meaning and affect, yet is tightly coupled with speaker identity. Existing methods either discard prosody for privacy or lack a principled mechanism to control the utility-privacy trade-off, operating at fixed design points. We propose DiffAnon, a diffusion-based anonymization method with classifier-free guidance (CFG) that provides explicit, continuous inference-time control over prosody preservation. DiffAnon refines acoustic detail over semantic embeddings of an RVQ codec, enabling smooth interpolation between anonymization strength and prosodic fidelity within a single model. To the best of our knowledge, it is the first voice anonymization framework to provide structured, interpolatable inference-time prosody control. Experiments demonstrate structured trade-off behavior, achieving strong utility while maintaining competitive privacy across controllable operating points.

Related papers

• DP-VOXLET: Provable Speaker Anonymization for Disentangled Speech Representations [2.565220169948739]
  Systems for speaker anonymization obfuscate the speaker of an utterance, while maintaining its original semantic contents and prosody.<n>Recent solutions for speaker anonymization rely on learned representations that disentangle an utterance into semantic contents and speaker properties.<n>We introduce speaker differential privacy, a formal definition of speaker anonymization based on the framework of differential privacy, and a mechanism for speaker anonymization that provably satisfies the definition.
  arXiv  Detail & Related papers  (2026-08-31T15:34:16Z)
• Semantic Leakage and Privacy Preservation in Relay-Assisted Semantic Communications [49.494000480610225]
  Semantic communication (SemCom) has emerged as a promising paradigm in which the transmission of task-relevant information is prioritized over raw data.<n>In this paper, the privacy implications of relay-assisted SemCom systems are studied, where the intermediate relay node operates directly on learned latent representations.<n>An adversarial training framework is proposed in which a strong, adaptively trained eavesdropper at the relay is explicitly accounted for.
  arXiv  Detail & Related papers  (2026-06-30T17:15:23Z)
• Mask-Free Privacy Extraction and Rewriting: A Domain-Aware Approach via Prototype Learning [13.300036863281996]
  We propose DAMPER (Domain-Aware Mask-free Privacy Extraction and Rewriting) for privacy-sensitive domains.<n>Damper operationalizes latent privacy semantics into compact Domain Privacy Prototypes via contrastive learning.<n>It integrates a sampling-based Exponential Mechanism to provide rigorous span-level Differential Privacy guarantees.
  arXiv  Detail & Related papers  (2026-04-11T10:21:00Z)
• You Only Anonymize What Is Not Intent-Relevant: Suppressing Non-Intent Privacy Evidence [25.1631668864743]
  We propose a utility-preserving anonymization approach that performs intent-conditioned exposure control.<n>IntentAnony models pragmatic intent and constructs privacy inference evidence chains.<n>Results show an approximately 30% improvement in the overall privacy--utility trade-off.
  arXiv  Detail & Related papers  (2026-01-07T07:54:23Z)
• VoxGuard: Evaluating User and Attribute Privacy in Speech via Membership Inference Attacks [51.68795949691009]
  We introduce VoxGuard, a framework grounded in differential privacy and membership inference.<n>For attributes, we show that simple transparent attacks recover gender and accent with near-perfect accuracy even after anonymization.<n>Our results demonstrate that EER substantially underestimates leakage, highlighting the need for low-FPR evaluation.
  arXiv  Detail & Related papers  (2025-09-22T20:57:48Z)
• Knockoffs Inference under Privacy Constraints [7.615990547453691]
  We propose a comprehensive framework for knockoff inference within the differential privacy paradigm.<n>Our proposed method guarantees robust privacy protection while preserving the exact FDR control entailed by the original model-X knockoff procedure.
  arXiv  Detail & Related papers  (2025-06-11T13:06:21Z)
• VoiceCloak: A Multi-Dimensional Defense Framework against Unauthorized Diffusion-based Voice Cloning [14.907575859145423]
  Diffusion Models (DMs) have achieved remarkable success in realistic voice cloning (VC)<n>DMs have been proven incompatible with proactive defenses due to intricate generative mechanisms of diffusion.<n>We introduce VoiceCloak, a multi-dimensional proactive defense framework with the goal of obfuscating speaker identity and degrading quality in potential unauthorized VC.
  arXiv  Detail & Related papers  (2025-05-18T09:58:48Z)
• Enhancing Privacy in Semantic Communication over Wiretap Channels leveraging Differential Privacy [51.028047763426265]
  Semantic communication (SemCom) improves transmission efficiency by focusing on task-relevant information.<n> transmitting semantic-rich data over insecure channels introduces privacy risks.<n>This paper proposes a novel SemCom framework that integrates differential privacy mechanisms to protect sensitive semantic features.
  arXiv  Detail & Related papers  (2025-04-23T08:42:44Z)
• Collaborative Inference over Wireless Channels with Feature Differential Privacy [57.68286389879283]
  Collaborative inference among multiple wireless edge devices has the potential to significantly enhance Artificial Intelligence (AI) applications. transmitting extracted features poses a significant privacy risk, as sensitive personal data can be exposed during the process. We propose a novel privacy-preserving collaborative inference mechanism, wherein each edge device in the network secures the privacy of extracted features before transmitting them to a central server for inference.
  arXiv  Detail & Related papers  (2024-10-25T18:11:02Z)
• Breaking the Communication-Privacy-Accuracy Tradeoff with $f$-Differential Privacy [51.11280118806893]
  We consider a federated data analytics problem in which a server coordinates the collaborative data analysis of multiple users with privacy concerns and limited communication capability. We study the local differential privacy guarantees of discrete-valued mechanisms with finite output space through the lens of $f$-differential privacy (DP) More specifically, we advance the existing literature by deriving tight $f$-DP guarantees for a variety of discrete-valued mechanisms.
  arXiv  Detail & Related papers  (2023-02-19T16:58:53Z)
• SPAct: Self-supervised Privacy Preservation for Action Recognition [73.79886509500409]
  Existing approaches for mitigating privacy leakage in action recognition require privacy labels along with the action labels from the video dataset. Recent developments of self-supervised learning (SSL) have unleashed the untapped potential of the unlabeled data. We present a novel training framework which removes privacy information from input video in a self-supervised manner without requiring privacy labels.
  arXiv  Detail & Related papers  (2022-03-29T02:56:40Z)
• A Shuffling Framework for Local Differential Privacy [40.92785300658643]
  ldp deployments are vulnerable to inference attacks as an adversary can link the noisy responses to their identity. An alternative model, shuffle DP, prevents this by shuffling the noisy responses uniformly at random. We show that systematic shuffling of the noisy responses can thwart specific inference attacks while retaining some meaningful data learnability.
  arXiv  Detail & Related papers  (2021-06-11T20:36:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.