Dhvani: A Weakly-supervised Phonemic Error Detection and Personalized Feedback System for Hindi

• URL: http://arxiv.org/abs/2506.02166v1
• Date: Mon, 02 Jun 2025 18:45:52 GMT
• Title: Dhvani: A Weakly-supervised Phonemic Error Detection and Personalized Feedback System for Hindi
• Authors: Arnav Rustagi, Satvik Bajpai, Nimrat Kaur, Siddharth Siddharth,
• Abstract summary: Computer-Assisted Pronunciation Training (CAPT) has been extensively studied for English.<n>There remains a critical gap in its application to Indian languages with a base of 1.5 billion speakers.<n>This paper proposes Dhvani -- a novel CAPT system for Hindi.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Computer-Assisted Pronunciation Training (CAPT) has been extensively studied for English. However, there remains a critical gap in its application to Indian languages with a base of 1.5 billion speakers. Pronunciation tools tailored to Indian languages are strikingly lacking despite the fact that millions learn them every year. With over 600 million speakers and being the fourth most-spoken language worldwide, improving Hindi pronunciation is a vital first step toward addressing this gap. This paper proposes 1) Dhvani -- a novel CAPT system for Hindi, 2) synthetic speech generation for Hindi mispronunciations, and 3) a novel methodology for providing personalized feedback to learners. While the system often interacts with learners using Devanagari graphemes, its core analysis targets phonemic distinctions, leveraging Hindi's highly phonetic orthography to analyze mispronounced speech and provide targeted feedback.

Related papers

• Kinship in Speech: Leveraging Linguistic Relatedness for Zero-Shot TTS in Indian Languages [6.74683227658822]
  India has 1369 languages, with 22 official using 13 scripts.<n>Our work focuses on zero-shot synthesis, particularly for languages whose scripts and phonotactics come from different families.<n>Intelligible and natural speech was generated for Sanskrit, Maharashtrian and Canara Konkani, Maithili and Kurukh.
  arXiv  Detail & Related papers  (2025-06-04T12:22:24Z)
• LAHAJA: A Robust Multi-accent Benchmark for Evaluating Hindi ASR Systems [16.143694951047024]
  We create a benchmark, LAHAJA, which contains read and extempore speech on a diverse set of topics and use cases. We evaluate existing open-source and commercial models on LAHAJA and find their performance to be poor. We train models using different datasets and find that our model trained on multilingual data with good speaker diversity outperforms existing models by a significant margin.
  arXiv  Detail & Related papers  (2024-08-21T08:51:00Z)
• Navigating Text-to-Image Generative Bias across Indic Languages [53.92640848303192]
  This research investigates biases in text-to-image (TTI) models for the Indic languages widely spoken across India. It evaluates and compares the generative performance and cultural relevance of leading TTI models in these languages against their performance in English.
  arXiv  Detail & Related papers  (2024-08-01T04:56:13Z)
• Low-Resource Counterspeech Generation for Indic Languages: The Case of Bengali and Hindi [11.117463901375602]
  We bridge the gap for low-resource languages such as Bengali and Hindi. We create a benchmark dataset of 5,062 abusive speech/counterspeech pairs. We observe that the monolingual setup yields the best performance.
  arXiv  Detail & Related papers  (2024-02-11T18:09:50Z)
• Scaling Speech Technology to 1,000+ Languages [66.31120979098483]
  The Massively Multilingual Speech (MMS) project increases the number of supported languages by 10-40x, depending on the task. Main ingredients are a new dataset based on readings of publicly available religious texts. We built pre-trained wav2vec 2.0 models covering 1,406 languages, a single multilingual automatic speech recognition model for 1,107 languages, speech synthesis models for the same number of languages, and a language identification model for 4,017 languages.
  arXiv  Detail & Related papers  (2023-05-22T22:09:41Z)
• An Investigation of Indian Native Language Phonemic Influences on L2 English Pronunciations [5.3956335232250385]
  Growing number of L2 English speakers in India reinforces need to study accents and L1-L2 interactions. We investigate the accents of Indian English (IE) speakers and report in detail our observations, both specific and common to all regions. We demonstrate the influence of 18 Indian languages on IE by comparing the native language pronunciations with IE pronunciations obtained jointly from existing literature studies and phonetically annotated speech of 80 speakers.
  arXiv  Detail & Related papers  (2022-12-19T07:41:39Z)
• DDSupport: Language Learning Support System that Displays Differences and Distances from Model Speech [16.82591185507251]
  We propose a new language learning support system that calculates speech scores and detects mispronunciations by beginners. The proposed system uses deep learning--based speech processing to display the pronunciation score of the learner's speech and the difference/distance between the learner's and a group of models' pronunciation.
  arXiv  Detail & Related papers  (2022-12-08T05:49:15Z)
• Ceasing hate withMoH: Hate Speech Detection in Hindi-English Code-Switched Language [2.9926023796813728]
  This work focuses on analyzing hate speech in Hindi-English code-switched language. To contain the structure of data, we developed MoH or Map Only Hindi, which means "Love" in Hindi. MoH pipeline consists of language identification, Roman to Devanagari Hindi transliteration using a knowledge base of Roman Hindi words.
  arXiv  Detail & Related papers  (2021-10-18T15:24:32Z)
• Phoneme Recognition through Fine Tuning of Phonetic Representations: a Case Study on Luhya Language Varieties [77.2347265289855]
  We focus on phoneme recognition using Allosaurus, a method for multilingual recognition based on phonetic annotation. To evaluate in a challenging real-world scenario, we curate phone recognition datasets for Bukusu and Saamia, two varieties of the Luhya language cluster of western Kenya and eastern Uganda. We find that fine-tuning of Allosaurus, even with just 100 utterances, leads to significant improvements in phone error rates.
  arXiv  Detail & Related papers  (2021-04-04T15:07:55Z)
• Speaker Independent and Multilingual/Mixlingual Speech-Driven Talking Head Generation Using Phonetic Posteriorgrams [58.617181880383605]
  In this work, we propose a novel approach using phonetic posteriorgrams. Our method doesn't need hand-crafted features and is more robust to noise compared to recent approaches. Our model is the first to support multilingual/mixlingual speech as input with convincing results.
  arXiv  Detail & Related papers  (2020-06-20T16:32:43Z)
• That Sounds Familiar: an Analysis of Phonetic Representations Transfer Across Languages [72.9927937955371]
  We use the resources existing in other languages to train a multilingual automatic speech recognition model. We observe significant improvements across all languages in the multilingual setting, and stark degradation in the crosslingual setting. Our analysis uncovered that even the phones that are unique to a single language can benefit greatly from adding training data from other languages.
  arXiv  Detail & Related papers  (2020-05-16T22:28:09Z)
• Towards Zero-shot Learning for Automatic Phonemic Transcription [82.9910512414173]
  A more challenging problem is to build phonemic transcribers for languages with zero training data. Our model is able to recognize unseen phonemes in the target language without any training data. It achieves 7.7% better phoneme error rate on average over a standard multilingual model.
  arXiv  Detail & Related papers  (2020-02-26T20:38:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.