CLewR: Curriculum Learning with Restarts for Machine Translation Preference Learning

• URL: http://arxiv.org/abs/2601.05858v1
• Date: Fri, 09 Jan 2026 15:34:31 GMT
• Title: CLewR: Curriculum Learning with Restarts for Machine Translation Preference Learning
• Authors: Alexandra Dragomir, Florin Brad, Radu Tudor Ionescu,
• Abstract summary: Large language models (LLMs) have demonstrated competitive performance in zero-shot multilingual machine translation (MT)<n>Some follow-up works further improved MT performance via preference optimization, but they leave a key aspect largely underexplored: the order in which data samples are given during training.<n>We address this topic by integrating curriculum learning into various state-of-the-art preference optimization algorithms to boost MT performance.
• Score: 61.359141900956125
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) have demonstrated competitive performance in zero-shot multilingual machine translation (MT). Some follow-up works further improved MT performance via preference optimization, but they leave a key aspect largely underexplored: the order in which data samples are given during training. We address this topic by integrating curriculum learning into various state-of-the-art preference optimization algorithms to boost MT performance. We introduce a novel curriculum learning strategy with restarts (CLewR), which reiterates easy-to-hard curriculum multiple times during training to effectively mitigate the catastrophic forgetting of easy examples. We demonstrate consistent gains across several model families (Gemma2, Qwen2.5, Llama3.1) and preference optimization techniques. We publicly release our code at https://github.com/alexandra-dragomir/CLewR.

Related papers

• Understanding In-Context Machine Translation for Low-Resource Languages: A Case Study on Manchu [53.437954702561065]
  In-context machine translation (MT) with large language models (LLMs) is a promising approach for low-resource MT.<n>This study systematically investigates how each type of resource, e.g., dictionary, grammar book, and retrieved parallel examples, affect the translation performance.<n>Our results indicate that high-quality dictionaries and good parallel examples are very helpful, while grammars hardly help.
  arXiv  Detail & Related papers  (2025-02-17T14:53:49Z)
• Context-aware Prompt Tuning: Advancing In-Context Learning with Adversarial Methods [69.36397993451742]
  This work introduces Context-aware Prompt Tuning (CPT), a method inspired by ICL, PT, and adversarial attacks. We modify specific context tokens, considering the unique structure of input and output formats. Inspired by adversarial attacks, we adjust the input based on the labels present in the context, focusing on minimizing, rather than maximizing, the loss.
  arXiv  Detail & Related papers  (2024-10-22T17:45:47Z)
• Making LLMs Better Many-to-Many Speech-to-Text Translators with Curriculum Learning [32.883836078329665]
  Multimodal Large Language Models (MLLMs) have achieved significant success in Speech-to-Text Translation (S2TT) tasks.<n>We propose a three-stage curriculum learning strategy that leverages the machine translation capabilities of large language models and adapts them to S2TT tasks.<n> Experimental results show that the proposed strategy achieves state-of-the-art average performance in $15times14$ language pairs.
  arXiv  Detail & Related papers  (2024-09-29T01:48:09Z)
• From Good to Best: Two-Stage Training for Cross-lingual Machine Reading Comprehension [51.953428342923885]
  We develop a two-stage approach to enhance the model performance. The first stage targets at recall: we design a hard-learning (HL) algorithm to maximize the likelihood that the top-k predictions contain the accurate answer. The second stage focuses on precision: an answer-aware contrastive learning mechanism is developed to learn the fine difference between the accurate answer and other candidates.
  arXiv  Detail & Related papers  (2021-12-09T07:31:15Z)
• Learning Kernel-Smoothed Machine Translation with Retrieved Examples [30.17061384497846]
  Existing non-parametric approaches that retrieve similar examples from a database to guide the translation process are promising but are prone to overfit the retrieved examples. We propose to learn Kernel-Smoothed Translation with Example Retrieval (KSTER), an effective approach to adapt neural machine translation models online.
  arXiv  Detail & Related papers  (2021-09-21T06:42:53Z)
• Netmarble AI Center's WMT21 Automatic Post-Editing Shared Task Submission [6.043109546012043]
  This paper describes Netmarble's submission to WMT21 Automatic Post-Editing (APE) Shared Task for the English-German language pair. Facebook Fair's WMT19 news translation model was chosen to engage the large and powerful pre-trained neural networks. For better performance, we leverage external translations as augmented machine translation (MT) during the post-training and fine-tuning.
  arXiv  Detail & Related papers  (2021-09-14T08:21:18Z)
• Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information [72.2412707779571]
  mRASP is an approach to pre-train a universal multilingual neural machine translation model. We carry out experiments on 42 translation directions across a diverse setting, including low, medium, rich resource, and as well as transferring to exotic language pairs.
  arXiv  Detail & Related papers  (2020-10-07T03:57:54Z)
• Recipes for Adapting Pre-trained Monolingual and Multilingual Models to Machine Translation [50.0258495437314]
  We investigate the benefits and drawbacks of freezing parameters, and adding new ones, when fine-tuning a pre-trained model on Machine Translation (MT) For BART we get the best performance by freezing most of the model parameters, and adding extra positional embeddings. For mBART we match or outperform the performance of naive fine-tuning for most language pairs with the encoder, and most of the decoder, frozen.
  arXiv  Detail & Related papers  (2020-04-30T16:09:22Z)
• Balancing Training for Multilingual Neural Machine Translation [130.54253367251738]
  multilingual machine translation (MT) models can translate to/from multiple languages. Standard practice is to up-sample less resourced languages to increase representation. We propose a method that instead automatically learns how to weight training data through a data scorer.
  arXiv  Detail & Related papers  (2020-04-14T18:23:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.