Reasoning Before Translation: Enhancing Legal Machine Translation with Structured Reasoning

• URL: http://arxiv.org/abs/2607.19181v1
• Date: Tue, 21 Jul 2026 15:15:56 GMT
• Title: Reasoning Before Translation: Enhancing Legal Machine Translation with Structured Reasoning
• Abstract summary: Legal machine translation (NMT) in the legal domain is a linguistically and conceptually demanding task.<n>Recent emergence of reasoning-capable language models opens new possibilities for tackling such challenges.<n>We evaluate small language models such as Qwen3.5 4B, Qwen3.5 9B, and Gemma 3 12B enhanced with various re-training paradigms.
• Score: 0.4266292701694338
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neural machine translation (NMT) in the legal domain is a linguistically and conceptually demanding task, primarily due to the complexity of legal language and the high level of precision it requires. The recent emergence of reasoning-capable language models opens new possibilities for tackling such challenges. They add to a set of other previously proposed techniques to enhance the translation quality, which includes supervised fine-tuning and reinforcement learning. In this work, we perform a comparison between these various approaches. More particularly, we evaluate small language models such as Qwen3.5 4B, Qwen3.5 9B, and Gemma 3 12B enhanced with various re-training paradigms and compare their performances against frontier reasoning models. We focus on the Swiss legal system, which -- with its unique multilingual statutes -- offers a particularly challenging testbed for reasoning-augmented models. Our results show that the quality of small ``base'' models can be greatly enhanced, and that reinforcement learning with verifiable rewards can be applied to NMT in the legal domain and surpasses the translation quality of supervised fine-tuning. The performance of enhanced small models is close to the one of state-of-the-art reasoning models yet remains inferior. We also note that re-training paradigms yield diminishing returns as model size increase. The code and models are publicly available at https://github.com/aixiuxiuxiu/Legal-MT-SFT-RL.

Related papers

• Unlocking Reasoning Capability on Machine Translation in Large Language Models [57.60641851466707]
  Reasoning-oriented large language models (RLMs) achieve strong gains on tasks such as mathematics and coding by generating explicit intermediate reasoning.<n>We systematically evaluate several open- and closed-weights RLMs on the WMT24++ benchmark.<n>We find that enabling explicit reasoning consistently degrades translation quality across languages and models.
  arXiv  Detail & Related papers  (2026-02-16T14:05:59Z)
• Mitigating Catastrophic Forgetting in Language Transfer via Model Merging [16.845734486667226]
  Branch-and-Merge (BaM) is a new adaptation method based on iteratively merging multiple models. BaM is based on the insight that this yields lower magnitude but higher quality weight changes. We demonstrate in an empirical study on Bulgarian and German that BaM can significantly reduce forgetting while matching or even improving target domain performance.
  arXiv  Detail & Related papers  (2024-07-11T17:32:40Z)
• Observational Scaling Laws and the Predictability of Language Model Performance [51.2336010244645]
  We propose an observational approach that bypasses model training and instead builds scaling laws from 100 publically available models. We show that several emergent phenomena follow a smooth, sigmoidal behavior and are predictable from small models. We show how to predict the impact of post-training interventions like Chain-of-Thought and Self-Consistency as language model capabilities continue to improve.
  arXiv  Detail & Related papers  (2024-05-17T17:49:44Z)
• The Power of Question Translation Training in Multilingual Reasoning: Broadened Scope and Deepened Insights [108.40766216456413]
  We propose a question alignment framework to bridge the gap between large language models' English and non-English performance. Experiment results show it can boost multilingual performance across diverse reasoning scenarios, model families, and sizes. We analyze representation space, generated response and data scales, and reveal how question translation training strengthens language alignment within LLMs.
  arXiv  Detail & Related papers  (2024-05-02T14:49:50Z)
• Low-resource neural machine translation with morphological modeling [3.3721926640077804]
  Morphological modeling in neural machine translation (NMT) is a promising approach to achieving open-vocabulary machine translation. We propose a framework-solution for modeling complex morphology in low-resource settings. We evaluate our proposed solution on Kinyarwanda - English translation using public-domain parallel text.
  arXiv  Detail & Related papers  (2024-04-03T01:31:41Z)
• On the Analysis of Cross-Lingual Prompt Tuning for Decoder-based Multilingual Model [49.81429697921861]
  We study the interaction between parameter-efficient fine-tuning (PEFT) and cross-lingual tasks in multilingual autoregressive models. We show that prompt tuning is more effective in enhancing the performance of low-resource languages than fine-tuning.
  arXiv  Detail & Related papers  (2023-11-14T00:43:33Z)
• Neural Machine Translation Models Can Learn to be Few-shot Learners [2.2999148299770042]
  We show that a much smaller model can be trained to perform in-context learning (ICL) With this capacity for ICL, the model can take advantage of relevant few-shot examples to adapt its output towards the domain. Our approach allows efficient batch inference on a mix of domains and outperforms state-of-the-art baselines in terms of both translation quality and immediate adaptation rate.
  arXiv  Detail & Related papers  (2023-09-15T17:44:21Z)
• A Template-based Method for Constrained Neural Machine Translation [100.02590022551718]
  We propose a template-based method that can yield results with high translation quality and match accuracy while keeping the decoding speed. The generation and derivation of the template can be learned through one sequence-to-sequence training framework. Experimental results show that the proposed template-based methods can outperform several representative baselines in lexically and structurally constrained translation tasks.
  arXiv  Detail & Related papers  (2022-05-23T12:24:34Z)
• LegaLMFiT: Efficient Short Legal Text Classification with LSTM Language Model Pre-Training [0.0]
  Large Transformer-based language models such as BERT have led to broad performance improvements on many NLP tasks. In legal NLP, BERT-based models have led to new state-of-the-art results on multiple tasks. We show that lightweight LSTM-based Language Models are able to capture enough information from a small legal text pretraining corpus and achieve excellent performance on short legal text classification tasks.
  arXiv  Detail & Related papers  (2021-09-02T14:45:04Z)
• Comparing the Performance of NLP Toolkits and Evaluation measures in Legal Tech [0.0]
  We compare and analyze the pretrained Neural Language Models, XLNet (autoregressive), and BERT (autoencoder) on the Legal Tasks. XLNet Model performs better on our Sequence Classification task of Legal Opinions Classification, whereas BERT produces better results on the NER task. We use domain-specific pretraining and additional legal vocabulary to adapt BERT Model further to the Legal Domain.
  arXiv  Detail & Related papers  (2021-03-12T11:06:32Z)
• Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation [81.7786241489002]
  Massively multilingual models for neural machine translation (NMT) are theoretically attractive, but often underperform bilingual models and deliver poor zero-shot translations. We argue that multilingual NMT requires stronger modeling capacity to support language pairs with varying typological characteristics. We propose random online backtranslation to enforce the translation of unseen training language pairs.
  arXiv  Detail & Related papers  (2020-04-24T17:21:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.