L-Proto: Language-Aware Episodic Prototypical Training for Multilingual Speaker Verification

• URL: http://arxiv.org/abs/2606.17416v1
• Date: Tue, 16 Jun 2026 01:57:41 GMT
• Title: L-Proto: Language-Aware Episodic Prototypical Training for Multilingual Speaker Verification
• Abstract summary: We propose L-Proto, a language-aware episodic training strategy that constructs language-consistent episodes.<n> Experiments on the TidyVoice Challenge benchmark demonstrate consistent performance improvements over conventional fine-tuning and random episodic sampling.
• Score: 49.128847336227636
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multilingual speaker verification remains challenging because language-dependent acoustic variability causes speaker identity to become entangled with linguistic characteristics, degrading generalization across languages. In multilingual training, embeddings often encode language cues with speaker identity, causing speakers to form language-specific clusters. We propose L-Proto, a language-aware episodic prototypical training strategy that constructs language-consistent episodes. By sampling speakers from a single language per episode, L-Proto reduces language-driven variation during training and encourages embeddings to focus more directly on speaker identity. Experiments on the TidyVoice Challenge benchmark demonstrate consistent performance improvements over conventional fine-tuning and random episodic sampling across multiple backbone architectures.

Related papers

• When Meanings Meet: Investigating the Emergence and Quality of Shared Concept Spaces during Multilingual Language Model Training [57.230355403478995]
  We investigate the development of language-agnostic concept spaces during pretraining of EuroLLM.<n>We find that shared concept spaces emerge early and continue to refine, but that alignment with them is language-dependent.<n>In contrast to prior work, our fine-grained manual analysis reveals that some apparent gains in translation quality reflect shifts in behavior.
  arXiv  Detail & Related papers  (2026-01-30T11:23:01Z)
• PART: Progressive Alignment Representation Training for Multilingual Speech-To-Text with LLMs [58.2469845374385]
  We introduce Progressive Alignment Representation Training (PART)<n>PART is a multi-stage and multi-task framework that separates within-language from cross-language alignment.<n>Experiments on CommonVoice 15, Fleurs, Wenetspeech, and CoVoST2 show that PART surpasses conventional approaches.
  arXiv  Detail & Related papers  (2025-09-24T03:54:14Z)
• LASPA: Language Agnostic Speaker Disentanglement with Prefix-Tuned Cross-Attention [2.199918533021483]
  The overlap between vocal traits such as accent, vocal anatomy, and a language's phonetic structure complicates separating linguistic and speaker information.<n>Disentangling these components can significantly improve speaker recognition accuracy.<n>We propose a novel disentanglement learning strategy that integrates joint learning through prefix-tuned cross-attention.
  arXiv  Detail & Related papers  (2025-06-02T10:59:31Z)
• Whisper Speaker Identification: Leveraging Pre-Trained Multilingual Transformers for Robust Speaker Embeddings [0.0]
  We propose WSI (Whisper Speaker Identification), a framework that repurposes the Whisper automatic speech recognition model pre trained on extensive multilingual data.<n>By capitalizing on Whisper language-agnostic acoustic representations, our approach effectively distinguishes speakers across diverse languages.
  arXiv  Detail & Related papers  (2025-03-13T15:11:28Z)
• CrossSpeech++: Cross-lingual Speech Synthesis with Decoupled Language and Speaker Generation [25.82932373649325]
  CrossSpeech++ is a method to disentangle language and speaker information.<n>It significantly improves the quality of cross-lingual speech synthesis.<n>We conduct extensive experiments using various metrics, and demonstrate that CrossSpeech++ achieves significant improvements.
  arXiv  Detail & Related papers  (2024-12-28T06:32:49Z)
• Mitigating the Linguistic Gap with Phonemic Representations for Robust Cross-lingual Transfer [26.014079273740485]
  Approaches to improving multilingual language understanding often struggle with significant performance gaps between high-resource and low-resource languages. We present experiments on three representative cross-lingual tasks on 12 languages in total. Phonemic representations exhibit higher similarities between languages compared to orthographic representations.
  arXiv  Detail & Related papers  (2024-02-22T04:41:52Z)
• Fine-Tuned Self-Supervised Speech Representations for Language Diarization in Multilingual Code-Switched Speech [4.39549503760707]
  We develop a continuous multilingual language diarizer using fine-tuned speech representations extracted from a large self-supervised architecture (WavLM) We experiment with a code-switched corpus consisting of five South African languages (isiZulu, isiXa, Setswana, Sesotho and English)
  arXiv  Detail & Related papers  (2023-12-15T09:40:41Z)
• ERNIE-SAT: Speech and Text Joint Pretraining for Cross-Lingual Multi-Speaker Text-to-Speech [58.93395189153713]
  We extend the pretraining method for cross-lingual multi-speaker speech synthesis tasks. We propose a speech-text joint pretraining framework, where we randomly mask the spectrogram and the phonemes. Our model shows great improvements over speaker-embedding-based multi-speaker TTS methods.
  arXiv  Detail & Related papers  (2022-11-07T13:35:16Z)
• Analyzing the Mono- and Cross-Lingual Pretraining Dynamics of Multilingual Language Models [73.11488464916668]
  This study investigates the dynamics of the multilingual pretraining process. We probe checkpoints taken from throughout XLM-R pretraining, using a suite of linguistic tasks. Our analysis shows that the model achieves high in-language performance early on, with lower-level linguistic skills acquired before more complex ones.
  arXiv  Detail & Related papers  (2022-05-24T03:35:00Z)
• Discovering Representation Sprachbund For Multilingual Pre-Training [139.05668687865688]
  We generate language representation from multilingual pre-trained models and conduct linguistic analysis. We cluster all the target languages into multiple groups and name each group as a representation sprachbund. Experiments are conducted on cross-lingual benchmarks and significant improvements are achieved compared to strong baselines.
  arXiv  Detail & Related papers  (2021-09-01T09:32:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.