DRT: Deep Reasoning Translation via Long Chain-of-Thought

• URL: http://arxiv.org/abs/2412.17498v3
• Date: Mon, 10 Feb 2025 11:35:28 GMT
• Title: DRT: Deep Reasoning Translation via Long Chain-of-Thought
• Authors: Jiaan Wang, Fandong Meng, Yunlong Liang, Jie Zhou,
• Abstract summary: In this paper, we introduce DRT, an attempt to bring the success of long CoT to neural machine translation (MT)<n>We first mine sentences containing similes or metaphors from existing literature books, and then develop a multi-agent framework to translate these sentences via long thought.<n>Using Qwen2.5 and LLama-3.1 as the backbones, DRT models can learn the thought process during machine translation.
• Score: 89.48208612476068
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, O1-like models have emerged as representative examples, illustrating the effectiveness of long chain-of-thought (CoT) in reasoning tasks such as math and coding tasks. In this paper, we introduce DRT, an attempt to bring the success of long CoT to neural machine translation (MT). Specifically, in view of the literature books that might involve similes and metaphors, translating these texts to a target language is very difficult in practice due to cultural differences. In such cases, literal translation often fails to convey the intended meaning effectively. Even for professional human translators, considerable thought must be given to preserving semantics throughout the translation process. To simulate LLMs' long thought ability in MT, we first mine sentences containing similes or metaphors from existing literature books, and then develop a multi-agent framework to translate these sentences via long thought. In the multi-agent framework, a translator is used to iteratively translate the source sentence under the suggestions provided by an advisor. To ensure the effectiveness of the long thoughts, an evaluator is also employed to quantify the translation quality in each round. In this way, we collect tens of thousands of long-thought MT data, which is used to train our DRT. Using Qwen2.5 and LLama-3.1 as the backbones, DRT models can learn the thought process during machine translation, and outperform vanilla LLMs as well as LLMs which are simply fine-tuning on the paired sentences without long thought, showing its effectiveness.

Related papers

• Deep Reasoning Translation via Reinforcement Learning [77.41383117199227]
  We introduce DeepTrans, a deep reasoning translation model that learns free translation via reinforcement learning. Given the source sentences, the reward model teaches the deep translation model how to think and free-translate them during reinforcement learning. Experimental results show that DeepTrans improves performance by 16.3% in literature translation.
  arXiv  Detail & Related papers  (2025-04-14T12:40:39Z)
• Compositional Translation: A Novel LLM-based Approach for Low-resource Machine Translation [20.704153242284114]
  Machine Translation has been shown to benefit from in-context examples when they are semantically similar to the sentence to translate. We propose a new LLM-based translation paradigm, compositional translation, to replace naive few-shot MT with similarity-based demonstrations. Our intuition is that this approach should improve translation because these shorter phrases should be intrinsically easier to translate and easier to match with relevant examples.
  arXiv  Detail & Related papers  (2025-03-06T15:37:31Z)
• Lost in Literalism: How Supervised Training Shapes Translationese in LLMs [51.04435855143767]
  Large language models (LLMs) have achieved remarkable success in machine translation. However, translationese, characterized by overly literal and unnatural translations, remains a persistent challenge. We introduce methods to mitigate these biases, including polishing golden references and filtering unnatural training instances.
  arXiv  Detail & Related papers  (2025-03-06T12:14:45Z)
• TasTe: Teaching Large Language Models to Translate through Self-Reflection [82.83958470745381]
  Large language models (LLMs) have exhibited remarkable performance in various natural language processing tasks. We propose the TasTe framework, which stands for translating through self-reflection. The evaluation results in four language directions on the WMT22 benchmark reveal the effectiveness of our approach compared to existing methods.
  arXiv  Detail & Related papers  (2024-06-12T17:21:21Z)
• Agent-SiMT: Agent-assisted Simultaneous Machine Translation with Large Language Models [38.49925017512848]
  Simultaneous Machine Translation (SiMT) generates target translations while reading the source sentence. Existing SiMT methods generally adopt the traditional Transformer architecture, which concurrently determines the policy and generates translations. We introduce Agent-SiMT, a framework combining the strengths of Large Language Models (LLMs) and traditional SiMT methods.
  arXiv  Detail & Related papers  (2024-06-11T03:09:20Z)
• (Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Texts [52.18246881218829]
  We introduce a novel multi-agent framework based on large language models (LLMs) for literary translation, implemented as a company called TransAgents. To evaluate the effectiveness of our system, we propose two innovative evaluation strategies: Monolingual Human Preference (MHP) and Bilingual LLM Preference (BLP)
  arXiv  Detail & Related papers  (2024-05-20T05:55:08Z)
• Machine Translation Models are Zero-Shot Detectors of Translation Direction [46.41883195574249]
  Detecting the translation direction of parallel text has applications for machine translation training and evaluation, but also has forensic applications such as resolving plagiarism or forgery allegations. In this work, we explore an unsupervised approach to translation direction detection based on the simple hypothesis that $p(texttranslation|textoriginal)>p(textoriginal|texttranslation)$, motivated by the well-known simplification effect in translationese or machine-translationese.
  arXiv  Detail & Related papers  (2024-01-12T18:59:02Z)
• Towards Effective Disambiguation for Machine Translation with Large Language Models [65.80775710657672]
  We study the capabilities of large language models to translate "ambiguous sentences" Experiments show that our methods can match or outperform state-of-the-art systems such as DeepL and NLLB in four out of five language directions.
  arXiv  Detail & Related papers  (2023-09-20T22:22:52Z)
• Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Decoding [53.84948040596055]
  We introduce two related methods to mitigate failure cases with a modified decoding objective. Experiments on the massively multilingual models M2M-100 (418M) and SMaLL-100 show that these methods suppress hallucinations and off-target translations.
  arXiv  Detail & Related papers  (2023-09-13T17:15:27Z)
• MTUncertainty: Assessing the Need for Post-editing of Machine Translation Outputs by Fine-tuning OpenAI LLMs [6.822926897514793]
  TQE is critical in assessing both machine translation (MT) and human translation (HT) quality without reference translations. This work examines whether the state-of-the-art large language models (LLMs) can be used for this purpose. We take OpenAI models as the best state-of-the-art technology and approach TQE as a binary classification task.
  arXiv  Detail & Related papers  (2023-07-31T21:13:30Z)
• TIM: Teaching Large Language Models to Translate with Comparison [78.66926087162672]
  We propose a novel framework using examples in comparison to teach LLMs to learn translation. Our approach involves presenting the model with examples of correct and incorrect translations and using a preference loss to guide the model's learning. Our findings offer a new perspective on fine-tuning LLMs for translation tasks and provide a promising solution for generating high-quality translations.
  arXiv  Detail & Related papers  (2023-07-10T08:15:40Z)
• Do GPTs Produce Less Literal Translations? [20.095646048167612]
  Large Language Models (LLMs) have emerged as general-purpose language models capable of addressing many natural language generation or understanding tasks. We find that translations out of English (E-X) from GPTs tend to be less literal, while exhibiting similar or better scores on Machine Translation quality metrics.
  arXiv  Detail & Related papers  (2023-05-26T10:38:31Z)
• The RoyalFlush System for the WMT 2022 Efficiency Task [11.00644143928471]
  This paper describes the submission of the Royal neural machine translation system for the WMT 2022 translation efficiency task. Unlike the commonly used autoregressive translation system, we adopted a two-stage translation paradigm called Hybrid Regression Translation. Our fastest system reaches 6k+ words/second on the GPU latency setting, estimated to be about 3.1x faster than the last year's winner.
  arXiv  Detail & Related papers  (2022-12-03T05:36:10Z)
• Hybrid-Regressive Neural Machine Translation [11.634586560239404]
  We investigate how to combine the strengths of autoregressive and non-autoregressive translation paradigms better. We propose a new two-stage translation prototype called hybrid-regressive translation (HRT) HRT achieves the state-of-the-art BLEU score of 28.49 on the WMT En-De task and is at least 1.5x faster than AT, regardless of batch size and device.
  arXiv  Detail & Related papers  (2022-10-19T09:26:15Z)
• Rethinking Round-Trip Translation for Machine Translation Evaluation [44.83568796515321]
  We report the surprising finding that round-trip translation can be used for automatic evaluation without the references. We demonstrate the rectification is overdue as round-trip translation could benefit multiple machine translation evaluation tasks.
  arXiv  Detail & Related papers  (2022-09-15T15:06:20Z)
• BitextEdit: Automatic Bitext Editing for Improved Low-Resource Machine Translation [53.55009917938002]
  We propose to refine the mined bitexts via automatic editing. Experiments demonstrate that our approach successfully improves the quality of CCMatrix mined bitext for 5 low-resource language-pairs and 10 translation directions by up to 8 BLEU points.
  arXiv  Detail & Related papers  (2021-11-12T16:00:39Z)
• Non-Autoregressive Translation with Layer-Wise Prediction and Deep Supervision [33.04082398101807]
  Existing neural machine translation models, such as Transformer, achieve high performance, but they decode words one by one, which is inefficient. Recent non-autoregressive translation models speed up the inference, but their quality is still inferior. We propose DSLP, a highly efficient and high-performance model for machine translation.
  arXiv  Detail & Related papers  (2021-10-14T16:36:12Z)
• Unsupervised Bitext Mining and Translation via Self-trained Contextual Embeddings [51.47607125262885]
  We describe an unsupervised method to create pseudo-parallel corpora for machine translation (MT) from unaligned text. We use multilingual BERT to create source and target sentence embeddings for nearest-neighbor search and adapt the model via self-training. We validate our technique by extracting parallel sentence pairs on the BUCC 2017 bitext mining task and observe up to a 24.5 point increase (absolute) in F1 scores over previous unsupervised methods.
  arXiv  Detail & Related papers  (2020-10-15T14:04:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.