ViDia2Std: A Parallel Corpus and Methods for Low-Resource Vietnamese Dialect-to-Standard Translation

• URL: http://arxiv.org/abs/2603.10211v1
• Date: Tue, 10 Mar 2026 20:24:38 GMT
• Title: ViDia2Std: A Parallel Corpus and Methods for Low-Resource Vietnamese Dialect-to-Standard Translation
• Abstract summary: We introduce ViDia2Std, the first manually annotated parallel corpus for dialect-to-standard Vietnamese translation.<n>Unlike prior datasets, ViDia2Std includes diverse dialects from Central, Southern, and non-standard Northern regions.<n>We report agreement rates of 86% (North), 82% (Central), and 85% (South) for ViDia2Std.
• Score: 5.3220011447194215
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vietnamese exhibits extensive dialectal variation, posing challenges for NLP systems trained predominantly on standard Vietnamese. Such systems often underperform on dialectal inputs, especially from underrepresented Central and Southern regions. Previous work on dialect normalization has focused narrowly on Central-to-Northern dialect transfer using synthetic data and limited dialectal diversity. These efforts exclude Southern varieties and intra-regional variants within the North. We introduce ViDia2Std, the first manually annotated parallel corpus for dialect-to-standard Vietnamese translation covering all 63 provinces. Unlike prior datasets, ViDia2Std includes diverse dialects from Central, Southern, and non-standard Northern regions often absent from existing resources, making it the most dialectally inclusive corpus to date. The dataset consists of over 13,000 sentence pairs sourced from real-world Facebook comments and annotated by native speakers across all three dialect regions. To assess annotation consistency, we define a semantic mapping agreement metric that accounts for synonymous standard mappings across annotators. Based on this criterion, we report agreement rates of 86% (North), 82% (Central), and 85% (South). We benchmark several sequence-to-sequence models on ViDia2Std. mBART-large-50 achieves the best results (BLEU 0.8166, ROUGE-L 0.9384, METEOR 0.8925), while ViT5-base offers competitive performance with fewer parameters. ViDia2Std demonstrates that dialect normalization substantially improves downstream tasks, highlighting the need for dialect-aware resources in building robust Vietnamese NLP systems.

Related papers

• Poly-Dialectal Neural Machine Translation System for Bangla Regional Dialects [0.0]
  Regional dialectal variation poses a fundamental challenge to natural language processing (NLP) in Bangla.<n>Over 240 million speakers communicate across diverse regional variants that diverge significantly from Standard Colloquial Bangla.<n>We present a unified Poly-Dialectal Neural Machine Translation System capable of multi-directional translation across 12 Bangla regional dialects.
  arXiv  Detail & Related papers  (2026-08-12T12:55:54Z)
• How Robust Are LLMs to Vietnamese Dialects? [7.401730363695013]
  We present the first systematic evaluation of Large Language Models (LLMs) evaluating Vietnamese to Vietnamese dialect variation across multiple tasks.<n>VialectBench contains 400 Standard Vietnamese source instances and 2,400 human-written dialectal rewrites spanning emotion recognition (ER), natural language inference (NLI), question answering (QA), and multiple-choice question answering (MCQA)<n>Across ten instruction-tuned models, dialectal inputs reduce average performance by 2.82%, and no evaluated model is fully dialect-invariant.
  arXiv  Detail & Related papers  (2026-08-11T03:02:57Z)
• Phonetic Modeling of Dialectal Variation in Vietnamese Speech [8.447993256993428]
  Vietnamese exhibits substantial dialectal phonetic variation across Northern, Central, and Southern regions.<n>We propose a dialect-aware phonetic framework that explicitly models Vietnamese phonological structure and dialectal variation at both the vocabulary and decoding levels.
  arXiv  Detail & Related papers  (2026-05-23T08:00:26Z)
• DocAtlas: Multilingual Document Understanding Across 80+ Languages [58.715440331861295]
  We introduce DocAtlas, a framework that constructs high-fidelity OCR datasets and benchmarks covering 82 languages.<n>Our dual pipelines, differential rendering of native DOCX documents and synthetic-based generation for right-to-left scripts produce precise structural annotations.
  arXiv  Detail & Related papers  (2026-05-12T18:09:38Z)
• DIA-HARM: Dialectal Disparities in Harmful Content Detection Across 50 English Dialects [6.107850985025956]
  We present DIA-HARM, the first benchmark for evaluating disinformation detection across 50 English dialects.<n>Human-written dialectal content degrades detection by 1.4-3.6% F1, while AI-generated content remains stable.<n>We release the DIA-HARM framework, D3 corpus, and evaluation tools.
  arXiv  Detail & Related papers  (2026-04-07T01:43:48Z)
• INDIC DIALECT: A Multi Task Benchmark to Evaluate and Translate in Indian Language Dialects [10.663878830823043]
  In India, despite Hindi being the third most spoken language globally (over 600 million speakers), its numerous dialects remain underrepresented.<n>We introduce INDIC-DIALECT, a human-curated parallel corpus of 13k sentence pairs spanning 11 dialects and 2 languages: Hindi and Odia.
  arXiv  Detail & Related papers  (2026-01-15T13:40:27Z)
• DialectGen: Benchmarking and Improving Dialect Robustness in Multimodal Generation [111.94720088481614]
  Can multimodal generative models effectively produce content given dialectal textual input?<n>We construct a new large-scale benchmark spanning six common English dialects.<n>We design a general encoder-based mitigation strategy for multimodal generative models.
  arXiv  Detail & Related papers  (2025-10-16T17:56:55Z)
• Multi-Dialect Vietnamese: Task, Dataset, Baseline Models and Challenges [0.964547614383472]
  We introduce Vietnamese Multi-Dialect dataset, a novel comprehensive dataset capturing the rich diversity of 63 provincial dialects spoken across Vietnam. Our dataset comprises 102.56 hours of audio, consisting of approximately 19,000 utterances, and the associated transcripts contain over 1.2 million words.
  arXiv  Detail & Related papers  (2024-10-04T14:17:56Z)
• Voices Unheard: NLP Resources and Models for Yorùbá Regional Dialects [72.18753241750964]
  Yorub'a is an African language with roughly 47 million speakers. Recent efforts to develop NLP technologies for African languages have focused on their standard dialects. We take steps towards bridging this gap by introducing a new high-quality parallel text and speech corpus.
  arXiv  Detail & Related papers  (2024-06-27T22:38:04Z)
• DIALECTBENCH: A NLP Benchmark for Dialects, Varieties, and Closely-Related Languages [49.38663048447942]
  We propose DIALECTBENCH, the first-ever large-scale benchmark for NLP on varieties. This allows for a comprehensive evaluation of NLP system performance on different language varieties. We provide substantial evidence of performance disparities between standard and non-standard language varieties.
  arXiv  Detail & Related papers  (2024-03-16T20:18:36Z)
• Quantifying the Dialect Gap and its Correlates Across Languages [69.18461982439031]
  This work will lay the foundation for furthering the field of dialectal NLP by laying out evident disparities and identifying possible pathways for addressing them through mindful data collection.
  arXiv  Detail & Related papers  (2023-10-23T17:42:01Z)
• Multi-VALUE: A Framework for Cross-Dialectal English NLP [49.55176102659081]
  Multi- Dialect is a controllable rule-based translation system spanning 50 English dialects. Stress tests reveal significant performance disparities for leading models on non-standard dialects. We partner with native speakers of Chicano and Indian English to release new gold-standard variants of the popular CoQA task.
  arXiv  Detail & Related papers  (2022-12-15T18:17:01Z)
• Quantifying Language Variation Acoustically with Few Resources [4.162663632560141]
  Deep acoustic models might have learned linguistic information that transfers to low-resource languages. We compute pairwise pronunciation differences averaged over 10 words for over 100 individual dialects from four (regional) languages. Our results show that acoustic models outperform the (traditional) transcription-based approach without requiring phonetic transcriptions.
  arXiv  Detail & Related papers  (2022-05-05T15:00:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.