Evaluating Machine Translation Models for English-Hindi Language Pairs: A Comparative Analysis

• URL: http://arxiv.org/abs/2505.19604v1
• Date: Mon, 26 May 2025 07:15:06 GMT
• Title: Evaluating Machine Translation Models for English-Hindi Language Pairs: A Comparative Analysis
• Abstract summary: The study aims to provide insights into the effectiveness of different machine translation approaches in handling both general and specialized language domains.<n>Results indicate varying performance levels across different metrics, highlighting strengths and areas for improvement in current translation systems.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine translation has become a critical tool in bridging linguistic gaps, especially between languages as diverse as English and Hindi. This paper comprehensively evaluates various machine translation models for translating between English and Hindi. We assess the performance of these models using a diverse set of automatic evaluation metrics, both lexical and machine learning-based metrics. Our evaluation leverages an 18000+ corpus of English Hindi parallel dataset and a custom FAQ dataset comprising questions from government websites. The study aims to provide insights into the effectiveness of different machine translation approaches in handling both general and specialized language domains. Results indicate varying performance levels across different metrics, highlighting strengths and areas for improvement in current translation systems.

Related papers

• Systematic Analysis of Large Language Models and Transformer-Based Machine Translation for English-Tamil and Tamil-English Across Diverse Datasets [0.19116784879310025]
  This research presents the comprehensive evaluation of the performance of several multilingual translation models on English-Tamil and Tamil-English translations.<n>This study evaluates supervised NMT systems, NLLB and mBART, using both the BLEU and chrF metric, and examines how these systems perform on data of different quality levels and domains.
  arXiv  Detail & Related papers  (2026-07-27T14:54:01Z)
• XQ-MEval: A Dataset with Cross-lingual Parallel Quality for Benchmarking Translation Metrics [64.77152900881724]
  We propose XQ-MEval, a semi-automatically built dataset covering nine translation directions.<n>We inject MQM-defined errors into gold translations automatically, filter them by native speakers for reliability, and merge errors to generate pseudo translations with controllable quality.<n>Using XQ-MEval, our experiments on nine representative metrics reveal the inconsistency between averaging and human judgment.
  arXiv  Detail & Related papers  (2026-04-16T12:27:10Z)
• Evaluating Language Translation Models by Playing Telephone [5.02470728447561]
  We propose an unsupervised method to generate training data for translation evaluation over different document lengths and application domains.<n>We evaluate evaluation systems trained on texts mechanically generated using both model rotation and language translation approaches.
  arXiv  Detail & Related papers  (2025-09-23T22:01:52Z)
• Parallel Corpora for Machine Translation in Low-resource Indic Languages: A Comprehensive Review [2.377892000761193]
  This review provides a comprehensive overview of available parallel corpora for Indic languages.<n>We critically examine the challenges faced in corpus creation, including linguistic diversity, script variation, and data scarcity.<n>We outline future directions, including leveraging cross-lingual transfer learning, expanding multilingual datasets, and integrating multimodal resources to enhance translation quality.
  arXiv  Detail & Related papers  (2025-03-02T21:22:53Z)
• Sentiment Analysis Across Languages: Evaluation Before and After Machine Translation to English [0.0]
  This paper examines the performance of transformer models in Sentiment Analysis tasks across multilingual datasets and text that has undergone machine translation. By comparing the effectiveness of these models in different linguistic contexts, we gain insights into their performance variations and potential implications for sentiment analysis across diverse languages.
  arXiv  Detail & Related papers  (2024-05-05T10:52:09Z)
• Understanding Cross-Lingual Alignment -- A Survey [52.572071017877704]
  Cross-lingual alignment is the meaningful similarity of representations across languages in multilingual language models. We survey the literature of techniques to improve cross-lingual alignment, providing a taxonomy of methods and summarising insights from throughout the field.
  arXiv  Detail & Related papers  (2024-04-09T11:39:53Z)
• Decomposed Prompting for Machine Translation Between Related Languages using Large Language Models [55.35106713257871]
  We introduce DecoMT, a novel approach of few-shot prompting that decomposes the translation process into a sequence of word chunk translations. We show that DecoMT outperforms the strong few-shot prompting BLOOM model with an average improvement of 8 chrF++ scores across the examined languages.
  arXiv  Detail & Related papers  (2023-05-22T14:52:47Z)
• An Evaluation of Persian-English Machine Translation Datasets with Transformers [1.0742675209112622]
  This study collected and analysed the most popular and valuable parallel corpora, which were used for Persian-English translation. We fine-tuned and evaluated two state-of-the-art attention-based seq2seq models on each dataset separately.
  arXiv  Detail & Related papers  (2023-02-01T08:55:08Z)
• Extrinsic Evaluation of Machine Translation Metrics [78.75776477562087]
  It is unclear if automatic metrics are reliable at distinguishing good translations from bad translations at the sentence level. We evaluate the segment-level performance of the most widely used MT metrics (chrF, COMET, BERTScore, etc.) on three downstream cross-lingual tasks. Our experiments demonstrate that all metrics exhibit negligible correlation with the extrinsic evaluation of the downstream outcomes.
  arXiv  Detail & Related papers  (2022-12-20T14:39:58Z)
• FRMT: A Benchmark for Few-Shot Region-Aware Machine Translation [64.9546787488337]
  We present FRMT, a new dataset and evaluation benchmark for Few-shot Region-aware Machine Translation. The dataset consists of professional translations from English into two regional variants each of Portuguese and Mandarin Chinese.
  arXiv  Detail & Related papers  (2022-10-01T05:02:04Z)
• DivEMT: Neural Machine Translation Post-Editing Effort Across Typologically Diverse Languages [5.367993194110256]
  DivEMT is the first publicly available post-editing study of Neural Machine Translation (NMT) over a typologically diverse set of target languages. We assess the impact on translation productivity of two state-of-the-art NMT systems, namely: Google Translate and the open-source multilingual model mBART50.
  arXiv  Detail & Related papers  (2022-05-24T17:22:52Z)
• Does Summary Evaluation Survive Translation to Other Languages? [0.0]
  We translate an existing English summarization dataset, SummEval dataset, to four different languages. We analyze the scores from the automatic evaluation metrics in translated languages, as well as their correlation with human annotations in the source language.
  arXiv  Detail & Related papers  (2021-09-16T17:35:01Z)
• Curious Case of Language Generation Evaluation Metrics: A Cautionary Tale [52.663117551150954]
  A few popular metrics remain as the de facto metrics to evaluate tasks such as image captioning and machine translation. This is partly due to ease of use, and partly because researchers expect to see them and know how to interpret them. In this paper, we urge the community for more careful consideration of how they automatically evaluate their models.
  arXiv  Detail & Related papers  (2020-10-26T13:57:20Z)
• Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations [83.27475281544868]
  We use singular vector canonical correlation analysis to study what kind of information is induced from each source. We observe that our representations embed typology and strengthen correlations with language relationships. We then take advantage of our multi-view language vector space for multilingual machine translation, where we achieve competitive overall translation accuracy.
  arXiv  Detail & Related papers  (2020-04-30T16:25:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.