Phrase-Localized Language-Contrastive Guidance: Training-Free Localized Accent Control for Code-Switching Text-to-Speech

• URL: http://arxiv.org/abs/2609.01016v1
• Date: Tue, 01 Sep 2026 10:07:00 GMT
• Title: Phrase-Localized Language-Contrastive Guidance: Training-Free Localized Accent Control for Code-Switching Text-to-Speech
• Abstract summary: We propose a training-free inference framework that restores a native accent to code-switched phrases in text-to-speech.<n>We propose a self-attention probing technique that finds the phrase boundaries without external alignments.<n>Across diverse language pairs, LCG robustly increases the nativeness of the code-switched phrase while suppressing accent leakage.
• Score: 47.27636239794203
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current speech synthesis struggles with code-switching, which mixes a foreign language phrase into a primary language utterance, causing the phrase to be spoken with the primary language's accent rather than its native one. We propose Phrase-Localized Language-Contrastive Guidance (LCG), a training-free inference framework that restores a native accent to code-switched phrases in cross-lingual text-to-speech. LCG replaces the single language guidance applied across the whole utterance with a separate guidance for each region, so each part is guided by its own language. To choose where to apply this localized guidance, we propose a self-attention probing technique that finds the phrase boundaries without external alignments. Together, these components generate speech in which each region carries the accent of its own language, requiring no fine-tuning or auxiliary models. Across diverse language pairs, LCG robustly increases the nativeness of the code-switched phrase while suppressing accent leakage, and preserving overall speaker identity and naturalness.

Related papers

• L-Proto: Language-Aware Episodic Prototypical Training for Multilingual Speaker Verification [49.128847336227636]
  We propose L-Proto, a language-aware episodic training strategy that constructs language-consistent episodes.<n> Experiments on the TidyVoice Challenge benchmark demonstrate consistent performance improvements over conventional fine-tuning and random episodic sampling.
  arXiv  Detail & Related papers  (2026-06-16T01:57:41Z)
• KIT's Submission to Cross-Lingual Voice Cloning in IWSLT 2026 [61.29502937013759]
  Cross-lingual voice cloning aims to generate speech in a target language while preserving speaker identity from a source-language reference.<n>Key challenge is maintaining intelligibility and naturalness in the presence of accent variation and domain-specific vocabulary.<n>We build on a multilingual text-to-speech model, FishAudio-S2-Pro, and introduce language tag prompting to improve language control and reduce accent leakage.
  arXiv  Detail & Related papers  (2026-06-05T13:09:21Z)
• Fine-Tuned Self-Supervised Speech Representations for Language Diarization in Multilingual Code-Switched Speech [4.39549503760707]
  We develop a continuous multilingual language diarizer using fine-tuned speech representations extracted from a large self-supervised architecture (WavLM) We experiment with a code-switched corpus consisting of five South African languages (isiZulu, isiXa, Setswana, Sesotho and English)
  arXiv  Detail & Related papers  (2023-12-15T09:40:41Z)
• MUST&P-SRL: Multi-lingual and Unified Syllabification in Text and Phonetic Domains for Speech Representation Learning [0.76146285961466]
  We present a methodology for linguistic feature extraction, focusing on automatically syllabifying words in multiple languages. In both the textual and phonetic domains, our method focuses on the extraction of phonetic transcriptions from text, stress marks, and a unified automatic syllabification. The system was built with open-source components and resources.
  arXiv  Detail & Related papers  (2023-10-17T19:27:23Z)
• Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modeling [92.55131711064935]
  We propose a cross-lingual neural language model, VALL-E X, for cross-lingual speech synthesis. VALL-E X inherits strong in-context learning capabilities and can be applied for zero-shot cross-lingual text-to-speech synthesis and zero-shot speech-to-speech translation tasks. It can generate high-quality speech in the target language via just one speech utterance in the source language as a prompt while preserving the unseen speaker's voice, emotion, and acoustic environment.
  arXiv  Detail & Related papers  (2023-03-07T14:31:55Z)
• Towards Zero-Shot Code-Switched Speech Recognition [44.76492452463019]
  We seek to build effective code-switched (CS) automatic speech recognition systems (ASR) under the zero-shot setting. We propose to simplify each monolingual module by allowing them to transcribe all speech segments indiscriminately with a monolingual script. We apply this transliteration-based approach in an end-to-end differentiable neural network and demonstrate its efficacy for zero-shot CS ASR on Mandarin-English SEAME test sets.
  arXiv  Detail & Related papers  (2022-11-02T19:52:54Z)
• Code-Switching without Switching: Language Agnostic End-to-End Speech Translation [68.8204255655161]
  We treat speech recognition and translation as one unified end-to-end speech translation problem. By training LAST with both input languages, we decode speech into one target language, regardless of the input language.
  arXiv  Detail & Related papers  (2022-10-04T10:34:25Z)
• That Sounds Familiar: an Analysis of Phonetic Representations Transfer Across Languages [72.9927937955371]
  We use the resources existing in other languages to train a multilingual automatic speech recognition model. We observe significant improvements across all languages in the multilingual setting, and stark degradation in the crosslingual setting. Our analysis uncovered that even the phones that are unique to a single language can benefit greatly from adding training data from other languages.
  arXiv  Detail & Related papers  (2020-05-16T22:28:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.