StreamVoiceAnon+: Emotion-Preserving Streaming Speaker Anonymization via Frame-Level Acoustic Distillation

• URL: http://arxiv.org/abs/2603.06079v1
• Date: Fri, 06 Mar 2026 09:30:20 GMT
• Title: StreamVoiceAnon+: Emotion-Preserving Streaming Speaker Anonymization via Frame-Level Acoustic Distillation
• Abstract summary: We address the challenge of preserving emotional content in streaming speaker anonymization (SA)<n>We propose supervised finetuning with neutral-emotion utterance pairs from the same speaker, combined with frame-level emotion distillation on acoustic token hidden states.<n>On the VoicePrivacy 2024 protocol, our approach achieves a 49.2% UAR (emotion preservation) with 5.77% WER (intelligibility)
• Score: 56.49717639074325
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We address the challenge of preserving emotional content in streaming speaker anonymization (SA). Neural audio codec language models trained for audio continuation tend to degrade source emotion: content tokens discard emotional information, and the model defaults to dominant acoustic patterns rather than preserving paralinguistic attributes. We propose supervised finetuning with neutral-emotion utterance pairs from the same speaker, combined with frame-level emotion distillation on acoustic token hidden states. All modifications are confined to finetuning, which takes less than 2 hours on 4 GPUs and adds zero inference latency overhead, while maintaining a competitive 180ms streaming latency. On the VoicePrivacy 2024 protocol, our approach achieves a 49.2% UAR (emotion preservation) with 5.77% WER (intelligibility), a +24% relative UAR improvement over the baseline (39.7%->49.2%) and +10% over the emotion-prompt variant (44.6% UAR), while maintaining strong privacy (EER 49.0%). Demo and code are available: https://anonymous3842031239.github.io/

Related papers

• Low Resource Multimodal Translation of Nepali Spoken Words into Emotion-Conditioned Sign Language Avatars [0.0]
  NEST-V1 is a proof-of-concept framework that generates emotion-conditioned Nepali Sign Language avatars from spoken input.<n>Our preliminary results indicate the viability of real-time, emotionally expressive sign language communication systems for the hearing-impaired community.
  arXiv  Detail & Related papers  (2026-05-04T15:17:49Z)
• PHONOS: PHOnetic Neutralization for Online Streaming Applications [4.7828228833826145]
  We present PHONOS, a streaming module for real-time speaker anonymization.<n>PHONOS neutralizes non-native accent to sound native-like.<n>Our evaluations show an 81% reduction in non-native accent confidence.
  arXiv  Detail & Related papers  (2026-03-27T21:24:18Z)
• Privacy-Preserving End-to-End Full-Duplex Speech Dialogue Models [62.35104196350636]
  We show that SALM-Duplex leakage persists across all layers with SALM-Duplex showing stronger leakage in early layers while Moshi leaks uniformly, and that Linkability rises sharply within the first few turns.<n>We propose two streaming anonymization setups using StreamVoiceAnon: a front-end (AnonW2W) and a feature-domain replacement (AnonW2F)
  arXiv  Detail & Related papers  (2026-03-09T10:01:24Z)
• Stream-Voice-Anon: Enhancing Utility of Real-Time Speaker Anonymization via Neural Audio Codec and Language Models [51.7170633585748]
  Stream-Voice-Anon adapts modern causal LM-based NAC architectures specifically for streaming speaker anonymization.<n>Our anonymization approach incorporates pseudo-speaker representation sampling, a speaker embedding mixing and diverse prompt selection strategies.<n>Under the VoicePrivacy 2024 Challenge protocol, Stream-Voice-Anon achieves substantial improvements in intelligibility.
  arXiv  Detail & Related papers  (2026-01-20T13:23:44Z)
• HLTCOE JHU Submission to the Voice Privacy Challenge 2024 [31.94758615908198]
  We present a number of systems for the Voice Privacy Challenge. We find that while voice conversion systems better preserve emotional content, they struggle to conceal speaker identity in semi-white-box attack scenarios. We propose a random admixture system which seeks to balance out the strengths and weaknesses of the two categories of systems.
  arXiv  Detail & Related papers  (2024-09-13T15:29:37Z)
• A Cross-Modal Approach to Silent Speech with LLM-Enhanced Recognition [0.0]
  Silent Speech Interfaces (SSIs) offer a noninvasive alternative to brain-computer interfaces for soundless verbal communication. We introduce Multimodal Orofacial Neural Audio (MONA), a system that leverages cross-modal alignment to train a multimodal model with a shared latent representation. To the best of our knowledge, this work represents the first instance where noninvasive silent speech recognition on an open vocabulary has cleared the threshold of 15% WER.
  arXiv  Detail & Related papers  (2024-03-02T21:15:24Z)
• Attention-based Interactive Disentangling Network for Instance-level Emotional Voice Conversion [81.1492897350032]
  Emotional Voice Conversion aims to manipulate a speech according to a given emotion while preserving non-emotion components. We propose an Attention-based Interactive diseNtangling Network (AINN) that leverages instance-wise emotional knowledge for voice conversion.
  arXiv  Detail & Related papers  (2023-12-29T08:06:45Z)
• Convoifilter: A case study of doing cocktail party speech recognition [59.80042864360884]
  The model can decrease ASR's word error rate (WER) from 80% to 26.4% through this approach. We openly share our pre-trained model to foster further research hf.co/nguyenvulebinh/voice-filter.
  arXiv  Detail & Related papers  (2023-08-22T12:09:30Z)
• Audio-Visual Speech Codecs: Rethinking Audio-Visual Speech Enhancement by Re-Synthesis [67.73554826428762]
  We propose a novel audio-visual speech enhancement framework for high-fidelity telecommunications in AR/VR. Our approach leverages audio-visual speech cues to generate the codes of a neural speech, enabling efficient synthesis of clean, realistic speech from noisy signals.
  arXiv  Detail & Related papers  (2022-03-31T17:57:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.