Evaluation of forced alignment of code-mixed speech: the case of Hindi-English

• URL: http://arxiv.org/abs/2607.25581v1
• Date: Tue, 28 Jul 2026 11:08:05 GMT
• Title: Evaluation of forced alignment of code-mixed speech: the case of Hindi-English
• Abstract summary: We evaluate forced alignment of Hindi-English code-mixed speech using the Montreal Forced Aligner.<n>We address two problems: free variation involving native vs non-native pairs and phonemic boundary detection for mid-utterance English words.
• Score: 5.706697761527132
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Code-mixed speech poses unique challenges to forced alignment: expanded inventories, orthographic errors, and speaker variation. We evaluate forced alignment of Hindi-English code-mixed speech using the Montreal Forced Aligner. We address 2 problems: (1) free variation involving native vs non-native pairs and (2) phonemic boundary detection for mid-utterance English words. Bootstrapping strategies substantially outperform unmodified lexicons. Acoustic models trained on sentence-level code-mixed data achieve a mean error of 4.15ms, ie. ten times lower than monolingual Hindi (38.18ms) or isolated English (37.58ms) alternatives. Principled lexicon design and code-mixed training data are both essential for reliable alignment of bilingual speech.

Related papers

• Evaluating Cross-lingual Knowledge Consistency in Code-Mixed vis-a-vis Indian Languages using IndicKLAR [11.507193494503754]
  We examine how knowledge recall consistency varies across the spectrum of English, code-mixed, and native Indian language inputs.<n>We find that the native-language accuracy gap to English can reach $sim$0.50, while code-mixed inputs close most of it.<n>Motivated by this, we evaluate several prompting strategies that vary in how language conversion is exposed.
  arXiv  Detail & Related papers  (2026-05-28T09:06:59Z)
• Neither Here Nor There: Cross-Lingual Representation Dynamics of Code-Mixed Text in Multilingual Encoders [0.9523331544264052]
  We construct a unified trilingual corpus of parallel English, Hindi, and Romanized code-mixed sentences.<n>We find that while standard models align English and Hindi well, code-mixed inputs remain loosely connected to either language.
  arXiv  Detail & Related papers  (2026-03-20T09:02:38Z)
• Simultaneous Speech-to-Speech Translation Without Aligned Data [52.467808474293605]
  Simultaneous speech translation requires translating source speech into a target language in real-time.<n>We propose Hibiki-Zero, which eliminates the need for word-level alignments entirely.<n>Hibiki-Zero achieves state-of-the-art performance in translation accuracy, latency, voice transfer, and naturalness across five X-to-English tasks.
  arXiv  Detail & Related papers  (2026-02-11T17:41:01Z)
• Languages in Multilingual Speech Foundation Models Align Both Phonetically and Semantically [58.019484208091534]
  Cross-lingual alignment in pretrained language models (LMs) has enabled efficient transfer in text-based LMs.<n>It remains an open question whether findings and methods from text-based cross-lingual alignment apply to speech.
  arXiv  Detail & Related papers  (2025-05-26T07:21:20Z)
• Modular Sentence Encoders: Separating Language Specialization from Cross-Lingual Alignment [50.80949663719335]
  multilingual sentence encoders (MSEs) are commonly obtained by training multilingual language models to map sentences from different languages into a shared semantic space.<n>MSEs are subject to curse of multilinguality, a loss of monolingual representational accuracy due to parameter sharing.<n>We train the cross-lingual adapters with two different types of data to resolve the conflicting requirements of different cross-lingual tasks.
  arXiv  Detail & Related papers  (2024-07-20T13:56:39Z)
• Multilingual self-supervised speech representations improve the speech recognition of low-resource African languages with codeswitching [65.74653592668743]
  Finetuning self-supervised multilingual representations reduces absolute word error rates by up to 20%. In circumstances with limited training data finetuning self-supervised representations is a better performing and viable solution.
  arXiv  Detail & Related papers  (2023-11-25T17:05:21Z)
• Reducing language context confusion for end-to-end code-switching automatic speech recognition [50.89821865949395]
  We propose a language-related attention mechanism to reduce multilingual context confusion for the E2E code-switching ASR model. By calculating the respective attention of multiple languages, our method can efficiently transfer language knowledge from rich monolingual data.
  arXiv  Detail & Related papers  (2022-01-28T14:39:29Z)
• Bridging the Modality Gap for Speech-to-Text Translation [57.47099674461832]
  End-to-end speech translation aims to translate speech in one language into text in another language via an end-to-end way. Most existing methods employ an encoder-decoder structure with a single encoder to learn acoustic representation and semantic information simultaneously. We propose a Speech-to-Text Adaptation for Speech Translation model which aims to improve the end-to-end model performance by bridging the modality gap between speech and text.
  arXiv  Detail & Related papers  (2020-10-28T12:33:04Z)
• Learning not to Discriminate: Task Agnostic Learning for Improving Monolingual and Code-switched Speech Recognition [12.354292498112347]
  We present further improvements over our previous work by using domain adversarial learning to train task models. Our proposed technique leads to reductions in Word Error Rates (WER) in monolingual and code-switched test sets across three language pairs.
  arXiv  Detail & Related papers  (2020-06-09T13:45:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.