The Effect of Alignment Objectives on Code-Switching Translation

• URL: http://arxiv.org/abs/2309.05044v1
• Date: Sun, 10 Sep 2023 14:46:31 GMT
• Title: The Effect of Alignment Objectives on Code-Switching Translation
• Authors: Mohamed Anwar
• Abstract summary: We are proposing a way of training a single machine translation model that is able to translate monolingual sentences from one language to another. This model can be considered a bilingual model in the human sense.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: One of the things that need to change when it comes to machine translation is the models' ability to translate code-switching content, especially with the rise of social media and user-generated content. In this paper, we are proposing a way of training a single machine translation model that is able to translate monolingual sentences from one language to another, along with translating code-switched sentences to either language. This model can be considered a bilingual model in the human sense. For better use of parallel data, we generated synthetic code-switched (CSW) data along with an alignment loss on the encoder to align representations across languages. Using the WMT14 English-French (En-Fr) dataset, the trained model strongly outperforms bidirectional baselines on code-switched translation while maintaining quality for non-code-switched (monolingual) data.

Related papers

• CoVoSwitch: Machine Translation of Synthetic Code-Switched Text Based on Intonation Units [0.0]
  We synthesize code-switching data by replacing intonation units detected through PSST. We evaluate the code-switching translation performance of two multilingual translation models, M2M-100 418M and NLLB-200 600M.
  arXiv  Detail & Related papers  (2024-07-19T13:26:35Z)
• Improving Zero-Shot Cross-Lingual Transfer via Progressive Code-Switching [35.27850496374157]
  Code-switching is a data augmentation scheme mixing words from multiple languages into source lingual text. We propose a Progressive Code-Switching (PCS) method to gradually generate moderately difficult code-switching examples for the model. Experiments show our model achieves state-of-the-art results on three different zero-shot cross-lingual transfer tasks across ten languages.
  arXiv  Detail & Related papers  (2024-06-19T09:06:24Z)
• Using Machine Translation to Augment Multilingual Classification [0.0]
  We explore the effects of using machine translation to fine-tune a multilingual model for a classification task across multiple languages. We show that translated data are of sufficient quality to tune multilingual classifiers and that this novel loss technique is able to offer some improvement over models tuned without it.
  arXiv  Detail & Related papers  (2024-05-09T00:31:59Z)
• Dual-Alignment Pre-training for Cross-lingual Sentence Embedding [79.98111074307657]
  We propose a dual-alignment pre-training (DAP) framework for cross-lingual sentence embedding. We introduce a novel representation translation learning (RTL) task, where the model learns to use one-side contextualized token representation to reconstruct its translation counterpart. Our approach can significantly improve sentence embedding.
  arXiv  Detail & Related papers  (2023-05-16T03:53:30Z)
• Breaking Down Multilingual Machine Translation [74.24795388967907]
  We show that multilingual training is beneficial to encoders in general, while it only benefits decoders for low-resource languages (LRLs) Our many-to-one models for high-resource languages and one-to-many models for LRLs outperform the best results reported by Aharoni et al.
  arXiv  Detail & Related papers  (2021-10-15T14:57:12Z)
• Using heterogeneity in semi-supervised transcription hypotheses to improve code-switched speech recognition [6.224255518500385]
  We show that monolingual data may be more closely matched to one of the languages in the code-switch pair. We propose a semi-supervised approach for code-switched ASR.
  arXiv  Detail & Related papers  (2021-06-14T18:39:18Z)
• Exploring Text-to-Text Transformers for English to Hinglish Machine Translation with Synthetic Code-Mixing [19.19256927651015]
  We describe models that convert monolingual English text into Hinglish (code-mixed Hindi and English) Given the recent success of pretrained language models, we also test the utility of two recent Transformer-based encoder-decoder models. Our models place first in the overall ranking of the English-Hinglish official shared task.
  arXiv  Detail & Related papers  (2021-05-18T19:50:25Z)
• XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders [89.0059978016914]
  We present XLM-T, which initializes the model with an off-the-shelf pretrained cross-lingual Transformer and fine-tunes it with multilingual parallel data. This simple method achieves significant improvements on a WMT dataset with 10 language pairs and the OPUS-100 corpus with 94 pairs.
  arXiv  Detail & Related papers  (2020-12-31T11:16:51Z)
• VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generation [77.82373082024934]
  We plug a cross-attention module into the Transformer encoder to explicitly build the interdependence between languages. It can effectively avoid the degeneration of predicting masked words only conditioned on the context in its own language. The proposed cross-lingual model delivers new state-of-the-art results on various cross-lingual understanding tasks of the XTREME benchmark.
  arXiv  Detail & Related papers  (2020-10-30T03:41:38Z)
• Beyond English-Centric Multilingual Machine Translation [74.21727842163068]
  We create a true Many-to-Many multilingual translation model that can translate directly between any pair of 100 languages. We build and open source a training dataset that covers thousands of language directions with supervised data, created through large-scale mining. Our focus on non-English-Centric models brings gains of more than 10 BLEU when directly translating between non-English directions while performing competitively to the best single systems of WMT.
  arXiv  Detail & Related papers  (2020-10-21T17:01:23Z)
• FILTER: An Enhanced Fusion Method for Cross-lingual Language Understanding [85.29270319872597]
  We propose an enhanced fusion method that takes cross-lingual data as input for XLM finetuning. During inference, the model makes predictions based on the text input in the target language and its translation in the source language. To tackle this issue, we propose an additional KL-divergence self-teaching loss for model training, based on auto-generated soft pseudo-labels for translated text in the target language.
  arXiv  Detail & Related papers  (2020-09-10T22:42:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.