CHILL at SemEval-2025 Task 2: You Can't Just Throw Entities and Hope -- Make Your LLM to Get Them Right

• URL: http://arxiv.org/abs/2506.13070v1
• Date: Mon, 16 Jun 2025 03:26:10 GMT
• Title: CHILL at SemEval-2025 Task 2: You Can't Just Throw Entities and Hope -- Make Your LLM to Get Them Right
• Authors: Jaebok Lee, Yonghyun Ryu, Seongmin Park, Yoonjung Choi,
• Abstract summary: We describe our approach for the SemEval 2025 Task 2 on Entity-Aware Machine Translation (EA-MT)<n>Our system aims to improve the accuracy of translating named entities by combining two key approaches: Retrieval Augmented Generation (RAG) and iterative self-refinement techniques using Large Language Models (LLMs)<n>A distinctive feature of our system is its self-evaluation mechanism, where the LLM assesses its own translations based on two key criteria: the accuracy of entity translations and overall translation quality.
• Score: 3.326216109891044
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: In this paper, we describe our approach for the SemEval 2025 Task 2 on Entity-Aware Machine Translation (EA-MT). Our system aims to improve the accuracy of translating named entities by combining two key approaches: Retrieval Augmented Generation (RAG) and iterative self-refinement techniques using Large Language Models (LLMs). A distinctive feature of our system is its self-evaluation mechanism, where the LLM assesses its own translations based on two key criteria: the accuracy of entity translations and overall translation quality. We demonstrate how these methods work together and effectively improve entity handling while maintaining high-quality translations.

Related papers

• Enhancing Entity Aware Machine Translation with Multi-task Learning [2.9611509639584312]
  We propose a method that applies multi-task learning to optimize the performance of the two subtasks named entity recognition and machine translation.<n>The result and analysis are performed on the dataset provided by the organizer of Task 2 of the SemEval 2025 competition.
  arXiv  Detail & Related papers  (2025-06-23T06:05:46Z)
• Improving Retrieval-Augmented Neural Machine Translation with Monolingual Data [9.67203800171351]
  In many settings, in-domain monolingual target-side corpora are often available.<n>This work explores ways to take advantage of such resources by retrieving relevant segments directly in the target language.<n>In experiments with two RANMT architectures, we first demonstrate the benefits of such cross-lingual objectives in a controlled setting.<n>We then showcase our method on a real-world set-up, where the target monolingual resources far exceed the amount of parallel data.
  arXiv  Detail & Related papers  (2025-04-30T15:41:03Z)
• HausaNLP at SemEval-2025 Task 2: Entity-Aware Fine-tuning vs. Prompt Engineering in Entity-Aware Machine Translation [2.17880235420183]
  This paper presents our findings for SemEval 2025 Task 2, a shared task on entity-aware machine translation (EA-MT)<n>The goal of this task is to develop translation models that can accurately translate English sentences into target languages.<n>In this paper, we describe the different systems we employed, detail our results, and discuss insights gained from our experiments.
  arXiv  Detail & Related papers  (2025-03-25T14:29:43Z)
• LLM-based Translation Inference with Iterative Bilingual Understanding [52.46978502902928]
  We propose a novel Iterative Bilingual Understanding Translation method based on the cross-lingual capabilities of large language models (LLMs)<n>The cross-lingual capability of LLMs enables the generation of contextual understanding for both the source and target languages separately.<n>The proposed IBUT outperforms several strong comparison methods.
  arXiv  Detail & Related papers  (2024-10-16T13:21:46Z)
• TasTe: Teaching Large Language Models to Translate through Self-Reflection [82.83958470745381]
  Large language models (LLMs) have exhibited remarkable performance in various natural language processing tasks. We propose the TasTe framework, which stands for translating through self-reflection. The evaluation results in four language directions on the WMT22 benchmark reveal the effectiveness of our approach compared to existing methods.
  arXiv  Detail & Related papers  (2024-06-12T17:21:21Z)
• TEaR: Improving LLM-based Machine Translation with Systematic Self-Refinement [26.26493253161022]
  Large Language Models (LLMs) have achieved impressive results in Machine Translation (MT) We introduce a systematic LLM-based self-refinement translation framework, named textbfTEaR.
  arXiv  Detail & Related papers  (2024-02-26T07:58:12Z)
• Towards Effective Disambiguation for Machine Translation with Large Language Models [65.80775710657672]
  We study the capabilities of large language models to translate "ambiguous sentences" Experiments show that our methods can match or outperform state-of-the-art systems such as DeepL and NLLB in four out of five language directions.
  arXiv  Detail & Related papers  (2023-09-20T22:22:52Z)
• The Devil is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation Evaluation [93.01964988474755]
  AutoMQM is a prompting technique which asks large language models to identify and categorize errors in translations. We study the impact of labeled data through in-context learning and finetuning. We then evaluate AutoMQM with PaLM-2 models, and we find that it improves performance compared to just prompting for scores.
  arXiv  Detail & Related papers  (2023-08-14T17:17:21Z)
• Exploring Human-Like Translation Strategy with Large Language Models [93.49333173279508]
  Large language models (LLMs) have demonstrated impressive capabilities in general scenarios. This work proposes the MAPS framework, which stands for Multi-Aspect Prompting and Selection. We employ a selection mechanism based on quality estimation to filter out noisy and unhelpful knowledge.
  arXiv  Detail & Related papers  (2023-05-06T19:03:12Z)
• Ensemble Fine-tuned mBERT for Translation Quality Estimation [0.0]
  In this paper, we discuss our submission to the WMT 2021 QE Shared Task. Our proposed system is an ensemble of multilingual BERT (mBERT)-based regression models. It demonstrates comparable performance with respect to the Pearson's correlation and beats the baseline system in MAE/ RMSE for several language pairs.
  arXiv  Detail & Related papers  (2021-09-08T20:13:06Z)
• On the Limitations of Cross-lingual Encoders as Exposed by Reference-Free Machine Translation Evaluation [55.02832094101173]
  Evaluation of cross-lingual encoders is usually performed either via zero-shot cross-lingual transfer in supervised downstream tasks or via unsupervised cross-lingual similarity. This paper concerns ourselves with reference-free machine translation (MT) evaluation where we directly compare source texts to (sometimes low-quality) system translations. We systematically investigate a range of metrics based on state-of-the-art cross-lingual semantic representations obtained with pretrained M-BERT and LASER. We find that they perform poorly as semantic encoders for reference-free MT evaluation and identify their two key limitations.
  arXiv  Detail & Related papers  (2020-05-03T22:10:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.