ViSpeechFormer: A Phonemic Approach for Vietnamese Automatic Speech Recognition

• URL: http://arxiv.org/abs/2602.10003v1
• Date: Tue, 10 Feb 2026 17:26:55 GMT
• Title: ViSpeechFormer: A Phonemic Approach for Vietnamese Automatic Speech Recognition
• Authors: Khoa Anh Nguyen, Long Minh Hoang, Nghia Hieu Nguyen, Luan Thanh Nguyen, Ngan Luu-Thuy Nguyen,
• Abstract summary: We propose ViSpeechFormer (textbfVietnamese textbfSpeech TranstextbfFormer), a phoneme-based approach for Vietnamese Automatic Speech Recognition (ASR)<n>Experiments on two publicly available Vietnamese ASR datasets show that ViSpeechFormer achieves strong performance, generalizes better to out-of-vocabulary words, and is less affected by training bias.
• Score: 7.250850162908686
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Vietnamese has a phonetic orthography, where each grapheme corresponds to at most one phoneme and vice versa. Exploiting this high grapheme-phoneme transparency, we propose ViSpeechFormer (\textbf{Vi}etnamese \textbf{Speech} Trans\textbf{Former}), a phoneme-based approach for Vietnamese Automatic Speech Recognition (ASR). To the best of our knowledge, this is the first Vietnamese ASR framework that explicitly models phonemic representations. Experiments on two publicly available Vietnamese ASR datasets show that ViSpeechFormer achieves strong performance, generalizes better to out-of-vocabulary words, and is less affected by training bias. This phoneme-based paradigm is also promising for other languages with phonetic orthographies. The code will be released upon acceptance of this paper.

Related papers

• POWSM: A Phonetic Open Whisper-Style Speech Foundation Model [50.73202227472358]
  POWSM is the first unified framework capable of jointly performing multiple phone-related tasks.<n>Our training data, code and models are released to foster open science.
  arXiv  Detail & Related papers  (2025-10-28T21:43:45Z)
• TSPC: A Two-Stage Phoneme-Centric Architecture for code-switching Vietnamese-English Speech Recognition [0.855801641444342]
  Code-switching (CS) presents a significant challenge for general Auto-Speech Recognition (ASR) systems.<n>Existing methods often fail to capture the subtle phonological shifts inherent in CS scenarios.<n>We propose a novel architecture for Vietnamese-English CS ASR, a Two-Stage Phoneme-Centric model (TSPC)
  arXiv  Detail & Related papers  (2025-09-07T09:19:03Z)
• Whisper based Cross-Lingual Phoneme Recognition between Vietnamese and English [0.0]
  Cross-lingual phoneme recognition has emerged as a significant challenge for accurate automatic speech recognition.<n>English features stress patterns and non-standard pronunciations that hinder phoneme alignment between the two languages.<n>We propose a novel bilingual speech recognition approach with two primary contributions.
  arXiv  Detail & Related papers  (2025-08-22T09:10:24Z)
• VALLR: Visual ASR Language Model for Lip Reading [28.561566996686484]
  Lip Reading, or Visual Automatic Speech Recognition, is a complex task requiring the interpretation of spoken language exclusively from visual cues.<n>We propose a novel two-stage, phoneme-centric framework for Visual Automatic Speech Recognition (V-ASR)<n>First, our model predicts a compact sequence of phonemes from visual inputs using a Video Transformer with a CTC head.<n>This phoneme output then serves as the input to a fine-tuned Large Language Model (LLM), which reconstructs coherent words and sentences.
  arXiv  Detail & Related papers  (2025-03-27T11:52:08Z)
• StyleDubber: Towards Multi-Scale Style Learning for Movie Dubbing [125.86266166482704]
  We propose StyleDubber, which switches dubbing learning from the frame level to phoneme level. It contains three main components: (1) A multimodal style adaptor operating at the phoneme level to learn pronunciation style from the reference audio, and generate intermediate representations informed by the facial emotion presented in the video; (2) An utterance-level style learning module, which guides both the mel-spectrogram decoding and the refining processes from the intermediate embeddings to improve the overall style expression; and (3) a phoneme-guided lip aligner to maintain lip sync.
  arXiv  Detail & Related papers  (2024-02-20T01:28:34Z)
• MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition [51.412413996510814]
  We propose MixSpeech, a cross-modality self-learning framework that utilizes audio speech to regularize the training of visual speech tasks. MixSpeech enhances speech translation in noisy environments, improving BLEU scores for four languages on AVMuST-TED by +1.4 to +4.2.
  arXiv  Detail & Related papers  (2023-03-09T14:58:29Z)
• MMSpeech: Multi-modal Multi-task Encoder-Decoder Pre-training for Speech Recognition [75.12948999653338]
  We propose a novel multi-task encoder-decoder pre-training framework (MMSpeech) for Mandarin automatic speech recognition (ASR) We employ a multi-task learning framework including five self-supervised and supervised tasks with speech and text data. Experiments on AISHELL-1 show that our proposed method achieves state-of-the-art performance, with a more than 40% relative improvement compared with other pre-training methods.
  arXiv  Detail & Related papers  (2022-11-29T13:16:09Z)
• VATLM: Visual-Audio-Text Pre-Training with Unified Masked Prediction for Speech Representation Learning [119.49605266839053]
  We propose a unified cross-modal representation learning framework VATLM (Visual-Audio-Text Language Model) The proposed VATLM employs a unified backbone network to model the modality-independent information. In order to integrate these three modalities into one shared semantic space, VATLM is optimized with a masked prediction task of unified tokens.
  arXiv  Detail & Related papers  (2022-11-21T09:10:10Z)
• SoundChoice: Grapheme-to-Phoneme Models with Semantic Disambiguation [10.016862617549991]
  This paper proposes SoundChoice, a novel Grapheme-to-Phoneme (G2P) architecture that processes entire sentences rather than operating at the word level. SoundChoice achieves a Phoneme Error Rate (PER) of 2.65% on whole-sentence transcription using data from LibriSpeech and Wikipedia.
  arXiv  Detail & Related papers  (2022-07-27T01:14:59Z)
• Phoneme Recognition through Fine Tuning of Phonetic Representations: a Case Study on Luhya Language Varieties [77.2347265289855]
  We focus on phoneme recognition using Allosaurus, a method for multilingual recognition based on phonetic annotation. To evaluate in a challenging real-world scenario, we curate phone recognition datasets for Bukusu and Saamia, two varieties of the Luhya language cluster of western Kenya and eastern Uganda. We find that fine-tuning of Allosaurus, even with just 100 utterances, leads to significant improvements in phone error rates.
  arXiv  Detail & Related papers  (2021-04-04T15:07:55Z)
• Augmenting Part-of-speech Tagging with Syntactic Information for Vietnamese and Chinese [0.32228025627337864]
  We implement the idea to improve word segmentation and part of speech tagging of the Vietnamese language by employing a simplified constituency. Our neural model for joint word segmentation and part-of-speech tagging has the architecture of the syllable-based constituency. This model can be augmented with predicted word boundary and part-of-speech tags by other tools.
  arXiv  Detail & Related papers  (2021-02-24T08:57:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.