Deep Reasoning Translation via Reinforcement Learning

• URL: http://arxiv.org/abs/2504.10187v1
• Date: Mon, 14 Apr 2025 12:40:39 GMT
• Title: Deep Reasoning Translation via Reinforcement Learning
• Authors: Jiaan Wang, Fandong Meng, Jie Zhou,
• Abstract summary: We introduce DeepTrans, a deep reasoning translation model that learns free translation via reinforcement learning.<n>Given the source sentences, the reward model teaches the deep translation model how to think and free-translate them during reinforcement learning.<n> Experimental results show that DeepTrans improves performance by 16.3% in literature translation.
• Score: 77.41383117199227
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, deep reasoning LLMs (e.g., OpenAI o1/o3 and DeepSeek-R1) have shown promising performance in various complex tasks. Free translation is an important and interesting task in the multilingual world, which requires going beyond word-for-word translation and taking cultural differences into account. This task is still under-explored in deep reasoning LLMs. In this paper, we introduce DeepTrans, a deep reasoning translation model that learns free translation via reinforcement learning. Specifically, we carefully build a reward model with pre-defined scoring criteria on both the translation results and the thought process. Given the source sentences, the reward model teaches the deep translation model how to think and free-translate them during reinforcement learning. In this way, training DeepTrans does not need any labeled translations, avoiding the human-intensive annotation or resource-intensive data synthesis. Experimental results show the effectiveness of DeepTrans. Using Qwen2.5-7B as the backbone, DeepTrans improves performance by 16.3% in literature translation, and outperforms strong deep reasoning baselines as well as baselines that are fine-tuned with synthesized data. Moreover, we summarize the failures and interesting findings during our RL exploration. We hope this work could inspire other researchers in free translation.

Related papers

• Lost in Literalism: How Supervised Training Shapes Translationese in LLMs [51.04435855143767]
  Large language models (LLMs) have achieved remarkable success in machine translation.<n>However, translationese, characterized by overly literal and unnatural translations, remains a persistent challenge.<n>We introduce methods to mitigate these biases, including polishing golden references and filtering unnatural training instances.
  arXiv  Detail & Related papers  (2025-03-06T12:14:45Z)
• DRT: Deep Reasoning Translation via Long Chain-of-Thought [89.48208612476068]
  In this paper, we introduce DRT, an attempt to bring the success of long CoT to neural machine translation (MT)<n>We first mine sentences containing similes or metaphors from existing literature books, and then develop a multi-agent framework to translate these sentences via long thought.<n>Using Qwen2.5 and LLama-3.1 as the backbones, DRT models can learn the thought process during machine translation.
  arXiv  Detail & Related papers  (2024-12-23T11:55:33Z)
• Multi-perspective Alignment for Increasing Naturalness in Neural Machine Translation [11.875491080062233]
  Neural machine translation (NMT) systems amplify lexical biases present in their training data, leading to artificially impoverished language in output translations.<n>We introduce a novel method that rewards both naturalness and content preservation.<n>We evaluate our method on English-to-Dutch literary translation, and find that our best model produces translations that are lexically richer and exhibit more properties of human-written language, without loss in translation accuracy.
  arXiv  Detail & Related papers  (2024-12-11T15:42:22Z)
• TasTe: Teaching Large Language Models to Translate through Self-Reflection [82.83958470745381]
  Large language models (LLMs) have exhibited remarkable performance in various natural language processing tasks. We propose the TasTe framework, which stands for translating through self-reflection. The evaluation results in four language directions on the WMT22 benchmark reveal the effectiveness of our approach compared to existing methods.
  arXiv  Detail & Related papers  (2024-06-12T17:21:21Z)
• Fine-Tuning Large Language Models to Translate: Will a Touch of Noisy Data in Misaligned Languages Suffice? [33.376648335299116]
  Large language models (LLMs) display strong translation capability after being fine-tuned on as few as 32 parallel sentences. LLMs with only English on the target side can lead to task misinterpretation, which hinders translation into non-English languages. synthesized data in an under-represented language has a less pronounced effect.
  arXiv  Detail & Related papers  (2024-04-22T12:21:12Z)
• Machine Translation Models are Zero-Shot Detectors of Translation Direction [46.41883195574249]
  Detecting the translation direction of parallel text has applications for machine translation training and evaluation, but also has forensic applications such as resolving plagiarism or forgery allegations.<n>In this work, we explore an unsupervised approach to translation direction detection based on the simple hypothesis that $p(texttranslation|textoriginal)>p(textoriginal|texttranslation)$, motivated by the well-known simplification effect in translationese or machine-translationese.
  arXiv  Detail & Related papers  (2024-01-12T18:59:02Z)
• Crossing the Threshold: Idiomatic Machine Translation through Retrieval Augmentation and Loss Weighting [66.02718577386426]
  We provide a simple characterization of idiomatic translation and related issues. We conduct a synthetic experiment revealing a tipping point at which transformer-based machine translation models correctly default to idiomatic translations. To improve translation of natural idioms, we introduce two straightforward yet effective techniques.
  arXiv  Detail & Related papers  (2023-10-10T23:47:25Z)
• Towards Debiasing Translation Artifacts [15.991970288297443]
  We propose a novel approach to reducing translationese by extending an established bias-removal technique. We use the Iterative Null-space Projection (INLP) algorithm, and show by measuring classification accuracy before and after debiasing, that translationese is reduced at both sentence and word level. To the best of our knowledge, this is the first study to debias translationese as represented in latent embedding space.
  arXiv  Detail & Related papers  (2022-05-16T21:46:51Z)
• Translation Artifacts in Cross-lingual Transfer Learning [51.66536640084888]
  We show that machine translation can introduce subtle artifacts that have a notable impact in existing cross-lingual models. In natural language inference, translating the premise and the hypothesis independently can reduce the lexical overlap between them. We also improve the state-of-the-art in XNLI for the translate-test and zero-shot approaches by 4.3 and 2.8 points, respectively.
  arXiv  Detail & Related papers  (2020-04-09T17:54:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.