Synthetic Data Generation and Joint Learning for Robust Code-Mixed Translation

• URL: http://arxiv.org/abs/2403.16771v2
• Date: Mon, 29 Apr 2024 20:45:53 GMT
• Title: Synthetic Data Generation and Joint Learning for Robust Code-Mixed Translation
• Authors: Kartik Kartik, Sanjana Soni, Anoop Kunchukuttan, Tanmoy Chakraborty, Md Shad Akhtar,
• Abstract summary: We tackle the problem of code-mixed (Hinglish and Bengalish) to English machine translation. We propose RCMT, a robust perturbation based joint-training model that learns to handle noise in the real-world code-mixed text. Our evaluation and comprehensive analyses demonstrate the superiority of RCMT over state-of-the-art code-mixed and robust translation methods.
• Score: 34.57825234659946
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The widespread online communication in a modern multilingual world has provided opportunities to blend more than one language (aka code-mixed language) in a single utterance. This has resulted a formidable challenge for the computational models due to the scarcity of annotated data and presence of noise. A potential solution to mitigate the data scarcity problem in low-resource setup is to leverage existing data in resource-rich language through translation. In this paper, we tackle the problem of code-mixed (Hinglish and Bengalish) to English machine translation. First, we synthetically develop HINMIX, a parallel corpus of Hinglish to English, with ~4.2M sentence pairs. Subsequently, we propose RCMT, a robust perturbation based joint-training model that learns to handle noise in the real-world code-mixed text by parameter sharing across clean and noisy words. Further, we show the adaptability of RCMT in a zero-shot setup for Bengalish to English translation. Our evaluation and comprehensive analyses qualitatively and quantitatively demonstrate the superiority of RCMT over state-of-the-art code-mixed and robust translation methods.

Related papers

• A Data Selection Approach for Enhancing Low Resource Machine Translation Using Cross-Lingual Sentence Representations [0.4499833362998489]
  This study focuses on the case of English-Marathi language pairs, where existing datasets are notably noisy. To mitigate the impact of data quality issues, we propose a data filtering approach based on cross-lingual sentence representations. Results demonstrate a significant improvement in translation quality over the baseline post-filtering with IndicSBERT.
  arXiv  Detail & Related papers  (2024-09-04T13:49:45Z)
• From Human Judgements to Predictive Models: Unravelling Acceptability in Code-Mixed Sentences [18.53327811304381]
  Modelling human judgements for the acceptability of code-mixed text can help in distinguishing natural code-mixed text. Cline is the largest of its kind with 16,642 sentences, consisting of samples sourced from two sources. Experiments using Cline demonstrate that simple Multilayer Perceptron (MLP) models trained solely on code-mixing metrics are outperformed by fine-tuned Multilingual Large Language Models (MLLMs)
  arXiv  Detail & Related papers  (2024-05-09T06:40:39Z)
• MoSECroT: Model Stitching with Static Word Embeddings for Crosslingual Zero-shot Transfer [50.40191599304911]
  We introduce MoSECroT Model Stitching with Static Word Embeddings for Crosslingual Zero-shot Transfer. In this paper, we present the first framework that leverages relative representations to construct a common space for the embeddings of a source language PLM and the static word embeddings of a target language. We show that although our proposed framework is competitive with weak baselines when addressing MoSECroT, it fails to achieve competitive results compared with some strong baselines.
  arXiv  Detail & Related papers  (2024-01-09T21:09:07Z)
• Mixed-Distil-BERT: Code-mixed Language Modeling for Bangla, English, and Hindi [0.0]
  We introduce Tri-Distil-BERT, a multilingual model pre-trained on Bangla, English, and Hindi, and Mixed-Distil-BERT, a model fine-tuned on code-mixed data. Our two-tiered pre-training approach offers efficient alternatives for multilingual and code-mixed language understanding.
  arXiv  Detail & Related papers  (2023-09-19T02:59:41Z)
• Simultaneous Machine Translation with Large Language Models [51.470478122113356]
  We investigate the possibility of applying Large Language Models to SimulMT tasks. We conducted experiments using the textttLlama2-7b-chat model on nine different languages from the MUST-C dataset. The results show that LLM outperforms dedicated MT models in terms of BLEU and LAAL metrics.
  arXiv  Detail & Related papers  (2023-09-13T04:06:47Z)
• Mitigating Data Imbalance and Representation Degeneration in Multilingual Machine Translation [103.90963418039473]
  Bi-ACL is a framework that uses only target-side monolingual data and a bilingual dictionary to improve the performance of the MNMT model. We show that Bi-ACL is more effective both in long-tail languages and in high-resource languages.
  arXiv  Detail & Related papers  (2023-05-22T07:31:08Z)
• Cross-Lingual Cross-Modal Retrieval with Noise-Robust Learning [25.230786853723203]
  We propose a noise-robust cross-lingual cross-modal retrieval method for low-resource languages. We use Machine Translation to construct pseudo-parallel sentence pairs for low-resource languages. We introduce a multi-view self-distillation method to learn noise-robust target-language representations.
  arXiv  Detail & Related papers  (2022-08-26T09:32:24Z)
• Exploring Text-to-Text Transformers for English to Hinglish Machine Translation with Synthetic Code-Mixing [19.19256927651015]
  We describe models that convert monolingual English text into Hinglish (code-mixed Hindi and English) Given the recent success of pretrained language models, we also test the utility of two recent Transformer-based encoder-decoder models. Our models place first in the overall ranking of the English-Hinglish official shared task.
  arXiv  Detail & Related papers  (2021-05-18T19:50:25Z)
• Continual Mixed-Language Pre-Training for Extremely Low-Resource Neural Machine Translation [53.22775597051498]
  We present a continual pre-training framework on mBART to effectively adapt it to unseen languages. Results show that our method can consistently improve the fine-tuning performance upon the mBART baseline. Our approach also boosts the performance on translation pairs where both languages are seen in the original mBART's pre-training.
  arXiv  Detail & Related papers  (2021-05-09T14:49:07Z)
• Cross-lingual Machine Reading Comprehension with Language Branch Knowledge Distillation [105.41167108465085]
  Cross-lingual Machine Reading (CLMRC) remains a challenging problem due to the lack of large-scale datasets in low-source languages. We propose a novel augmentation approach named Language Branch Machine Reading (LBMRC) LBMRC trains multiple machine reading comprehension (MRC) models proficient in individual language. We devise a multilingual distillation approach to amalgamate knowledge from multiple language branch models to a single model for all target languages.
  arXiv  Detail & Related papers  (2020-10-27T13:12:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.