The Price of Reasoning: Cost-Quality Tradeoffs in Reinforcement Learning for Neural Machine Translation

• URL: http://arxiv.org/abs/2607.19226v1
• Date: Tue, 21 Jul 2026 15:57:36 GMT
• Title: The Price of Reasoning: Cost-Quality Tradeoffs in Reinforcement Learning for Neural Machine Translation
• Abstract summary: We investigate the importance of including the model's reasoning trace in the generated responses during both training and inference.<n>Our experiments show that including the reasoning, specifically during inference, has a positive effect on the overall translation quality.
• Score: 0.1915265522996079
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reinforcement learning with verifiable rewards (RLVR) has been established as a viable paradigm for the post-training of Large Language Models (LLMs), including downstream tasks, such as Neural Machine Translation (NMT). With the latest research indicating that RLVR could be the preferred training method for translating legal documents due to the induced reasoning capabilities, it raises the question whether it is really attributed to the reasoning or more generally to the training paradigm. We investigate the importance of including the model's reasoning trace in the generated responses during both training and inference by systematically omitting it from one of the phases. Our experiments show that including the reasoning, specifically during inference, has a positive effect on the overall translation quality. Furthermore, we recognise that the reasoning leads to an increase in output tokens, hence we study the cost-quality tradeoff between the increased computational demands and the improved translation quality.

Related papers

• ReflectMT: Internalizing Reflection for Efficient and High-Quality Machine Translation [63.89730738735446]
  We propose ReflectMT, a two-stage reflection internalization algorithm for machine translation.<n>Our approach develops the model's "translate-reflect-refine" capability through reinforcement learning.<n> Experimental results on datasets such as WMT24 demonstrate that our model's first-pass translations during inference outperform multi-step reasoning LRMs.
  arXiv  Detail & Related papers  (2026-04-21T06:48:41Z)
• Unlocking Reasoning Capability on Machine Translation in Large Language Models [57.60641851466707]
  Reasoning-oriented large language models (RLMs) achieve strong gains on tasks such as mathematics and coding by generating explicit intermediate reasoning.<n>We systematically evaluate several open- and closed-weights RLMs on the WMT24++ benchmark.<n>We find that enabling explicit reasoning consistently degrades translation quality across languages and models.
  arXiv  Detail & Related papers  (2026-02-16T14:05:59Z)
• Efficient Reinforcement Learning for Large Language Models with Intrinsic Exploration [33.02780998281276]
  Reinforcement learning with verifiable rewards (RLVR) has improved the reasoning ability of large language models.<n>This study investigates how simply leveraging intrinsic data properties, almost free benefit during training, can improve data efficiency for RLVR.
  arXiv  Detail & Related papers  (2025-11-02T04:16:47Z)
• Parallel Scaling Law: Unveiling Reasoning Generalization through A Cross-Linguistic Perspective [52.452449102961225]
  This study proposes a novel cross-linguistic perspective to investigate reasoning generalization.<n>Our findings reveal that cross-lingual transferability varies significantly across initial model, target language, and training paradigm.<n>Our study challenges the assumption that LRM reasoning mirrors human cognition, providing critical insights for the development of more language-agnostic LRMs.
  arXiv  Detail & Related papers  (2025-10-02T17:49:49Z)
• Learning a Dense Reasoning Reward Model from Expert Demonstration via Inverse Reinforcement Learning [50.20267980386502]
  We learn a dense, token-level reward model for process supervision directly from expert demonstrations.<n>The learned reasoning reward serves two complementary roles: (i) it provides step-level feedback to optimise a reasoning policy during training; and (ii) it functions at inference as a critic to rerank sampled traces under fixed compute budgets.
  arXiv  Detail & Related papers  (2025-10-02T09:55:26Z)
• Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs [35.27561531876348]
  This paper systematically investigates the impact of Reinforcement Learning with Verifiable Rewards (RLVR) on Large Language Models (LLMs)<n>We show that RLVR can extend the reasoning boundary for both mathematical and coding tasks.<n>We present a theoretical framework explaining RLVR's incentive mechanism, demonstrating how it can encourage correct reasoning even when rewards are based solely on answer correctness.
  arXiv  Detail & Related papers  (2025-06-17T07:06:56Z)
• RIVAL: Reinforcement Learning with Iterative and Adversarial Optimization for Machine Translation [33.79108789619648]
  Large language models (LLMs) possess strong multilingual capabilities, and combining Reinforcement Learning from Human Feedback with translation tasks has shown great potential.<n>We observe that this paradigm performs unexpectedly poorly when applied to colloquial subtitle translation tasks.<n>We propose RIVAL, an adversarial training framework that formulates the process as a min-max game between the RM and the LLM.
  arXiv  Detail & Related papers  (2025-06-05T14:18:21Z)
• Fine-Grained Reward Optimization for Machine Translation using Error Severity Mappings [25.851419860597407]
  We propose a novel approach that leverages fine-grained, token-level quality assessments along with error severity levels usingReinforcement learning.<n>We conduct experiments on small and large translation datasets with standard encoder-decoder and large language models-based machine translation systems.<n>Our results show that training with token-level rewards improves translation quality across language pairs over baselines according to both automatic and human evaluation.
  arXiv  Detail & Related papers  (2024-11-08T21:55:37Z)
• Improving Machine Translation with Human Feedback: An Exploration of Quality Estimation as a Reward Model [75.66013048128302]
  In this work, we investigate the potential of employing the QE model as the reward model to predict human preferences for feedback training. We first identify the overoptimization problem during QE-based feedback training, manifested as an increase in reward while translation quality declines. To address the problem, we adopt a simple yet effective method that uses rules to detect the incorrect translations and assigns a penalty term to the reward scores of them.
  arXiv  Detail & Related papers  (2024-01-23T16:07:43Z)
• NoiER: An Approach for Training more Reliable Fine-TunedDownstream Task Models [54.184609286094044]
  We propose noise entropy regularisation (NoiER) as an efficient learning paradigm that solves the problem without auxiliary models and additional data. The proposed approach improved traditional OOD detection evaluation metrics by 55% on average compared to the original fine-tuned models.
  arXiv  Detail & Related papers  (2021-08-29T06:58:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.