Systematic Analysis of Large Language Models and Transformer-Based Machine Translation for English-Tamil and Tamil-English Across Diverse Datasets

• URL: http://arxiv.org/abs/2607.24515v2
• Date: Tue, 28 Jul 2026 17:28:27 GMT
• Title: Systematic Analysis of Large Language Models and Transformer-Based Machine Translation for English-Tamil and Tamil-English Across Diverse Datasets
• Abstract summary: This research presents the comprehensive evaluation of the performance of several multilingual translation models on English-Tamil and Tamil-English translations.<n>This study evaluates supervised NMT systems, NLLB and mBART, using both the BLEU and chrF metric, and examines how these systems perform on data of different quality levels and domains.
• Score: 0.19116784879310025
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The challenge of Machine Translation for low resource languages such as Tamil is primarily caused by the restricted amount of parallel data for these languages, as well as their substantial amount of domain variation and morphological complexity. This research presents the comprehensive evaluation of the performance of several multilingual translation models on English-Tamil and Tamil-English translations across multiple datasets: NTREX, EnTamV2, WikiMatrix and PMIndia. This study evaluates supervised NMT systems, NLLB and mBART, using both the BLEU and chrF metric, and examines how these systems perform on data of different quality levels and domains. This performs an attention-based analysis to increase model interpretability by visualising the alignments of tokens in an English source text and their Tamil translations and vice-versa to provide insight into how they make translations. This study also demonstrates that using in-context prompting can provide an excellent way to perform a few-shot translation of English to Tamil and Tamil-English using a Tamil capable TamilLaMA model, and compare this to supervised approaches qualitatively. These findings show that the quality of the datasets and their alignment with the domain will greatly affect the performance of the model, that attention-based mechanisms can aid in explain ability, and that few-shot large language models can still produce structurally coherent translations of Tamil.

Related papers

• Evaluating Machine Translation Models for English-Hindi Language Pairs: A Comparative Analysis [0.0]
  The study aims to provide insights into the effectiveness of different machine translation approaches in handling both general and specialized language domains.<n>Results indicate varying performance levels across different metrics, highlighting strengths and areas for improvement in current translation systems.
  arXiv  Detail & Related papers  (2025-05-26T07:15:06Z)
• Trans-Zero: Self-Play Incentivizes Large Language Models for Multilingual Translation Without Parallel Data [64.4458540273004]
  We propose a self-play framework that leverages only monolingual data and the intrinsic multilingual knowledge of Large Language Models (LLMs)<n>Experiments demonstrate that this approach not only matches the performance of models trained on large-scale parallel data but also excels in non-English translation directions.
  arXiv  Detail & Related papers  (2025-04-20T16:20:30Z)
• Contextual Cues in Machine Translation: Investigating the Potential of Multi-Source Input Strategies in LLMs and NMT Systems [2.512491726995032]
  We compare GPT-4o, a large language model, with a traditional multilingual neural machine translation (NMT) system.<n>Using intermediate language translations as contextual cues, we evaluate their effectiveness in enhancing English and Chinese translations into Portuguese.<n>Results suggest that contextual information significantly improves translation quality for domain-specific datasets and potentially for linguistically distant language pairs.
  arXiv  Detail & Related papers  (2025-03-10T11:23:44Z)
• P-MMEval: A Parallel Multilingual Multitask Benchmark for Consistent Evaluation of LLMs [84.24644520272835]
  We introduce P-MMEval, a large-scale benchmark covering effective fundamental and capability-specialized datasets.<n>P-MMEval delivers consistent language coverage across various datasets and provides parallel samples.<n>We conduct extensive experiments on representative multilingual model series to compare performances across models and tasks.
  arXiv  Detail & Related papers  (2024-11-14T01:29:36Z)
• Unified Model Learning for Various Neural Machine Translation [63.320005222549646]
  Existing machine translation (NMT) studies mainly focus on developing dataset-specific models. We propose a versatile'' model, i.e., the Unified Model Learning for NMT (UMLNMT) that works with data from different tasks. OurNMT results in substantial improvements over dataset-specific models with significantly reduced model deployment costs.
  arXiv  Detail & Related papers  (2023-05-04T12:21:52Z)
• DivEMT: Neural Machine Translation Post-Editing Effort Across Typologically Diverse Languages [5.367993194110256]
  DivEMT is the first publicly available post-editing study of Neural Machine Translation (NMT) over a typologically diverse set of target languages. We assess the impact on translation productivity of two state-of-the-art NMT systems, namely: Google Translate and the open-source multilingual model mBART50.
  arXiv  Detail & Related papers  (2022-05-24T17:22:52Z)
• Building Machine Translation Systems for the Next Thousand Languages [102.24310122155073]
  We describe results in three research domains: building clean, web-mined datasets for 1500+ languages, developing practical MT models for under-served languages, and studying the limitations of evaluation metrics for these languages. We hope that our work provides useful insights to practitioners working towards building MT systems for currently understudied languages, and highlights research directions that can complement the weaknesses of massively multilingual models in data-sparse settings.
  arXiv  Detail & Related papers  (2022-05-09T00:24:13Z)
• Towards the Next 1000 Languages in Multilingual Machine Translation: Exploring the Synergy Between Supervised and Self-Supervised Learning [48.15259834021655]
  We present a pragmatic approach towards building a multilingual machine translation model that covers hundreds of languages. We use a mixture of supervised and self-supervised objectives, depending on the data availability for different language pairs. We demonstrate that the synergy between these two training paradigms enables the model to produce high-quality translations in the zero-resource setting.
  arXiv  Detail & Related papers  (2022-01-09T23:36:44Z)
• Multilingual Neural Machine Translation:Can Linguistic Hierarchies Help? [29.01386302441015]
  Multilingual Neural Machine Translation (MNMT) trains a single NMT model that supports translation between multiple languages. The performance of an MNMT model is highly dependent on the type of languages used in training, as transferring knowledge from a diverse set of languages degrades the translation performance due to negative transfer. We propose a Hierarchical Knowledge Distillation (HKD) approach for MNMT which capitalises on language groups generated according to typological features and phylogeny of languages to overcome the issue of negative transfer.
  arXiv  Detail & Related papers  (2021-10-15T02:31:48Z)
• Exploring Pair-Wise NMT for Indian Languages [35.17470908190963]
  We show that the performance of these models can be significantly improved by using back-translation through a filtered back-translation process. The analysis in this paper suggests that this method can significantly improve a multilingual model's performance over its baseline.
  arXiv  Detail & Related papers  (2020-12-10T16:22:36Z)
• Beyond English-Centric Multilingual Machine Translation [74.21727842163068]
  We create a true Many-to-Many multilingual translation model that can translate directly between any pair of 100 languages. We build and open source a training dataset that covers thousands of language directions with supervised data, created through large-scale mining. Our focus on non-English-Centric models brings gains of more than 10 BLEU when directly translating between non-English directions while performing competitively to the best single systems of WMT.
  arXiv  Detail & Related papers  (2020-10-21T17:01:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.