Adding Robust Code-Switching Capabilities to High Performance Multilingual ASR

• URL: http://arxiv.org/abs/2606.21990v1
• Date: Sat, 20 Jun 2026 11:02:29 GMT
• Title: Adding Robust Code-Switching Capabilities to High Performance Multilingual ASR
• Abstract summary: We extend models to handle complex code-switching, including morphological variations across languages.<n>Our approach reduces transcription errors by 32.87% on code-switched words while improving overall WER by 5.31%, all while maintaining mono-lingual performance.
• Score: 61.09730719510015
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Code-switching (CSW) remains challenging for large multi-lingual ASR systems in real-world deployment. While fine-tuning on synthetic CSW data is possible, it generally degrades strong monolingual baselines. Our goal is to preserve these capabilities while extending models to handle complex code-switching, including morphological variations across languages. We propose Bayesian factorized adaptation, which learns to efficiently integrate switching-relevant knowledge into strong pretrained models without overwriting existing capabilities. Requiring only a small amount of synthetic data, our approach reduces transcription errors by 32.87% on code-switched words while improving overall WER by 5.31%, all while maintaining mono-lingual performance. Our results demonstrate that effective CSW adaptation depends more on knowledge integration than data complexity.

Related papers

• SAW: Stage-Aware Dynamic Weighting for Multi-Objective Reinforcement Learning in Large Language Models [66.92079232778707]
  We propose Stage-Aware Dynamic Weighting (SAW) as a lightweight, algorithm-agnostic dynamic weighting mechanism.<n>SAW reweights each dimension's reward or advantage contribution by its relative informativeness within the batch.<n> Experiments on tool-calling and text summarization tasks demonstrate that SAW consistently improves both training efficiency and final performance.
  arXiv  Detail & Related papers  (2026-06-05T10:00:19Z)
• Code-Switching Information Retrieval: Benchmarks, Analysis, and the Limits of Current Retrievers [32.12010196874932]
  We introduce CSR-L (Code-Switching Retrieval benchmark-Lite), constructing a dataset via human annotation to capture the authentic naturalness of mixed-language queries.<n>We show that code-switching acts as a fundamental performance bottleneck, degrading the effectiveness of even robust multilingual models.
  arXiv  Detail & Related papers  (2026-04-19T22:01:41Z)
• Beyond Quantity: Trajectory Diversity Scaling for Code Agents [51.71414642763219]
  Trajectory Diversity Scaling is a data synthesis framework for code agents that scales performance through diversity rather than raw volume.<n> TDScaling integrates four innovations: (1) a Business Cluster mechanism that captures real-service logical dependencies; (2) a blueprint-driven multi-agent paradigm that enforces trajectory coherence; and (3) an adaptive evolution mechanism that steers toward long-tail scenarios.
  arXiv  Detail & Related papers  (2026-02-03T07:43:03Z)
• Adapting Where It Matters: Depth-Aware Adaptation for Efficient Multilingual Speech Recognition in Low-Resource Languages [11.808922632545874]
  We analyze multilingual automatic speech recognition models and reveal a U-shaped adaptability pattern.<n>We propose DAMA, a Depth-Aware Model Adaptation framework that allocates adaptation capacity according to each layer's role.<n>Dama matches or surpasses state-of-the-art accuracy with 80% fewer trainable parameters, achieves a 29% error reduction under extreme data scarcity, and significantly improves memory, training time, and computational efficiency over baselines.
  arXiv  Detail & Related papers  (2026-02-01T04:18:31Z)
• How to Learn a New Language? An Efficient Solution for Self-Supervised Learning Models Unseen Languages Adaption in Low-Resource Scenario [72.02391485962127]
  Speech Self-Supervised Learning (SSL) models achieve impressive performance on Automatic Speech Recognition (ASR)<n>In low-resource language ASR, they encounter the domain mismatch problem between pre-trained and low-resource languages.<n>We extend a conventional efficient fine-tuning scheme based on the adapter to handle these issues.
  arXiv  Detail & Related papers  (2024-11-27T10:51:00Z)
• Balancing Accuracy and Efficiency in Multi-Turn Intent Classification for LLM-Powered Dialog Systems in Production [6.459396785817196]
  This paper presents two novel approaches to enhance scalability and reduce latency in production dialogue systems. First, we introduce Symbol Tuning, which simplifies intent labels to reduce task complexity and improve performance in multi-turn dialogues. Second, we propose C-LARA, a framework that employs LLMs for data augmentation and pseudo-labeling to generate synthetic multi-turn dialogues.
  arXiv  Detail & Related papers  (2024-11-19T07:48:35Z)
• Reducing language context confusion for end-to-end code-switching automatic speech recognition [50.89821865949395]
  We propose a language-related attention mechanism to reduce multilingual context confusion for the E2E code-switching ASR model. By calculating the respective attention of multiple languages, our method can efficiently transfer language knowledge from rich monolingual data.
  arXiv  Detail & Related papers  (2022-01-28T14:39:29Z)
• Code-Switching Text Augmentation for Multilingual Speech Processing [36.302629721413155]
  Code-switching in spoken content has enforced ASR systems to handle mixed input. Recent ASR studies showed the predominance of E2E-ASR using multilingual data to handle CS phenomena. We propose a methodology to augment the monolingual data for artificially generating spoken CS text to improve different speech modules.
  arXiv  Detail & Related papers  (2022-01-07T17:14:19Z)
• Arabic Code-Switching Speech Recognition using Monolingual Data [13.513655231184261]
  Code-switching in automatic speech recognition (ASR) is an important challenge due to globalization. Recent research in multilingual ASR shows potential improvement over monolingual systems. We study key issues related to multilingual modeling for ASR through a series of large-scale ASR experiments.
  arXiv  Detail & Related papers  (2021-07-04T08:40:49Z)
• Data Augmentation for End-to-end Code-switching Speech Recognition [54.0507000473827]
  Three novel approaches are proposed for code-switching data augmentation. Audio splicing with the existing code-switching data, and TTS with new code-switching texts generated by word translation or word insertion. Experiments on 200 hours Mandarin-English code-switching dataset show significant improvements on code-switching ASR individually.
  arXiv  Detail & Related papers  (2020-11-04T07:12:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.