Marathi To English Neural Machine Translation With Near Perfect Corpus And Transformers

• URL: http://arxiv.org/abs/2002.11643v1
• Date: Wed, 26 Feb 2020 17:18:49 GMT
• Title: Marathi To English Neural Machine Translation With Near Perfect Corpus And Transformers
• Authors: Swapnil Ashok Jadhav
• Abstract summary: Google, Bing, Facebook and Yandex are some of the very few companies which have built translation systems for few of the Indian languages. In this exercise, we trained and compared variety of Neural Machine Marathi to English Translators trained with BERT-tokenizer. We achieve better BLEU scores than Google on Tatoeba and Wikimedia open datasets.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: There have been very few attempts to benchmark performances of state-of-the-art algorithms for Neural Machine Translation task on Indian Languages. Google, Bing, Facebook and Yandex are some of the very few companies which have built translation systems for few of the Indian Languages. Among them, translation results from Google are supposed to be better, based on general inspection. Bing-Translator do not even support Marathi language which has around 95 million speakers and ranks 15th in the world in terms of combined primary and secondary speakers. In this exercise, we trained and compared variety of Neural Machine Marathi to English Translators trained with BERT-tokenizer by huggingface and various Transformer based architectures using Facebook's Fairseq platform with limited but almost correct parallel corpus to achieve better BLEU scores than Google on Tatoeba and Wikimedia open datasets.

Related papers

• CoSTA: Code-Switched Speech Translation using Aligned Speech-Text Interleaving [61.73180469072787]
  We focus on the problem of spoken translation (ST) of code-switched speech in Indian languages to English text. We present a new end-to-end model architecture COSTA that scaffolds on pretrained automatic speech recognition (ASR) and machine translation (MT) modules. COSTA significantly outperforms many competitive cascaded and end-to-end multimodal baselines by up to 3.5 BLEU points.
  arXiv  Detail & Related papers  (2024-06-16T16:10:51Z)
• Question answering using deep learning in low resource Indian language Marathi [0.0]
  We investigate different transformer models for creating a reading comprehension-based question answering system. We got the best accuracy in a MuRIL multilingual model with an EM score of 0.64 and F1 score of 0.74 by fine tuning the model on the Marathi dataset.
  arXiv  Detail & Related papers  (2023-09-27T16:53:11Z)
• Hindi to English: Transformer-Based Neural Machine Translation [0.0]
  We have developed a Machine Translation (NMT) system by training the Transformer model to translate texts from Indian Language Hindi to English. We implemented back-translation to augment the training data and for creating the vocabulary. This led us to achieve a state-of-the-art BLEU score of 24.53 on the test set of IIT Bombay English-Hindi Corpus.
  arXiv  Detail & Related papers  (2023-09-23T00:00:09Z)
• SeamlessM4T: Massively Multilingual & Multimodal Machine Translation [90.71078166159295]
  We introduce SeamlessM4T, a single model that supports speech-to-speech translation, speech-to-text translation, text-to-text translation, and automatic speech recognition for up to 100 languages. We developed the first multilingual system capable of translating from and into English for both speech and text. On FLEURS, SeamlessM4T sets a new standard for translations into multiple target languages, achieving an improvement of 20% BLEU over the previous SOTA in direct speech-to-text translation.
  arXiv  Detail & Related papers  (2023-08-22T17:44:18Z)
• The Effect of Normalization for Bi-directional Amharic-English Neural Machine Translation [53.907805815477126]
  This paper presents the first relatively large-scale Amharic-English parallel sentence dataset. We build bi-directional Amharic-English translation models by fine-tuning the existing Facebook M2M100 pre-trained model. The results show that the normalization of Amharic homophone characters increases the performance of Amharic-English machine translation in both directions.
  arXiv  Detail & Related papers  (2022-10-27T07:18:53Z)
• No Language Left Behind: Scaling Human-Centered Machine Translation [69.28110770760506]
  We create datasets and models aimed at narrowing the performance gap between low and high-resource languages. We propose multiple architectural and training improvements to counteract overfitting while training on thousands of tasks. Our model achieves an improvement of 44% BLEU relative to the previous state-of-the-art.
  arXiv  Detail & Related papers  (2022-07-11T07:33:36Z)
• Harnessing Cross-lingual Features to Improve Cognate Detection for Low-resource Languages [50.82410844837726]
  We demonstrate the use of cross-lingual word embeddings for detecting cognates among fourteen Indian languages. We evaluate our methods to detect cognates on a challenging dataset of twelve Indian languages. We observe an improvement of up to 18% points, in terms of F-score, for cognate detection.
  arXiv  Detail & Related papers  (2021-12-16T11:17:58Z)
• Improving Sentiment Analysis over non-English Tweets using Multilingual Transformers and Automatic Translation for Data-Augmentation [77.69102711230248]
  We propose the use of a multilingual transformer model, that we pre-train over English tweets and apply data-augmentation using automatic translation to adapt the model to non-English languages. Our experiments in French, Spanish, German and Italian suggest that the proposed technique is an efficient way to improve the results of the transformers over small corpora of tweets in a non-English language.
  arXiv  Detail & Related papers  (2020-10-07T15:44:55Z)
• HausaMT v1.0: Towards English-Hausa Neural Machine Translation [0.012691047660244334]
  We build a baseline model for English-Hausa machine translation. The Hausa language is the second largest Afro-Asiatic language in the world after Arabic.
  arXiv  Detail & Related papers  (2020-06-09T02:08:03Z)
• Neural Machine Translation for Low-Resourced Indian Languages [4.726777092009554]
  Machine translation is an effective approach to convert text to a different language without any human involvement. In this paper, we have applied NMT on two of the most morphological rich Indian languages, i.e. English-Tamil and English-Malayalam. We proposed a novel NMT model using Multihead self-attention along with pre-trained Byte-Pair-Encoded (BPE) and MultiBPE embeddings to develop an efficient translation system.
  arXiv  Detail & Related papers  (2020-04-19T17:29:34Z)
• Neural Machine Translation System of Indic Languages -- An Attention based Approach [0.5139874302398955]
  In India, almost all the languages are originated from their ancestral language - Sanskrit. In this paper, we have presented the neural machine translation system (NMT) that can efficiently translate Indic languages like Hindi and Gujarati.
  arXiv  Detail & Related papers  (2020-02-02T07:15:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.