Science Across Languages: Assessing LLM Multilingual Translation of Scientific Papers

• URL: http://arxiv.org/abs/2502.17882v1
• Date: Tue, 25 Feb 2025 06:08:48 GMT
• Title: Science Across Languages: Assessing LLM Multilingual Translation of Scientific Papers
• Authors: Hannah Calzi Kleidermacher, James Zou,
• Abstract summary: We leverage large language models (LLMs) to translate scientific articles.<n>We translate articles across multiple scientific disciplines into 28 languages.<n>Our benchmark results show an average performance of 95.9%.
• Score: 24.150250149027883
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scientific research is inherently global. However, the vast majority of academic journals are published exclusively in English, creating barriers for non-native-English-speaking researchers. In this study, we leverage large language models (LLMs) to translate published scientific articles while preserving their native JATS XML formatting, thereby developing a practical, automated approach for implementation by academic journals. Using our approach, we translate articles across multiple scientific disciplines into 28 languages. To evaluate translation accuracy, we introduce a novel question-and-answer (QA) benchmarking method, in which an LLM generates comprehension-based questions from the original text and then answers them based on the translated text. Our benchmark results show an average performance of 95.9%, showing that the key scientific details are accurately conveyed. In a user study, we translate the scientific papers of 15 researchers into their native languages, finding that the authors consistently found the translations to accurately capture the original information in their articles. Interestingly, a third of the authors found many technical terms "overtranslated," expressing a preference to keep terminology more familiar in English untranslated. Finally, we demonstrate how in-context learning techniques can be used to align translations with domain-specific preferences such as mitigating overtranslation, highlighting the adaptability and utility of LLM-driven scientific translation. The code and translated articles are available at https://hankleid.github.io/ProjectMundo.

Related papers

• Deep Reasoning Translation via Reinforcement Learning [77.41383117199227]
  We introduce DeepTrans, a deep reasoning translation model that learns free translation via reinforcement learning. Given the source sentences, the reward model teaches the deep translation model how to think and free-translate them during reinforcement learning. Experimental results show that DeepTrans improves performance by 16.3% in literature translation.
  arXiv  Detail & Related papers  (2025-04-14T12:40:39Z)
• Lost in Literalism: How Supervised Training Shapes Translationese in LLMs [51.04435855143767]
  Large language models (LLMs) have achieved remarkable success in machine translation. However, translationese, characterized by overly literal and unnatural translations, remains a persistent challenge. We introduce methods to mitigate these biases, including polishing golden references and filtering unnatural training instances.
  arXiv  Detail & Related papers  (2025-03-06T12:14:45Z)
• I Can't Share Code, but I need Translation -- An Empirical Study on Code Translation through Federated LLM [3.9373541926236766]
  This study demonstrates that participants can collaboratively develop a FedLLM for efficient code translation.<n>Our findings indicate that FedLLM offers a collaborative approach to code translation and could serve as a promising direction for future research in this field.
  arXiv  Detail & Related papers  (2025-01-10T05:43:36Z)
• Dictionary Insertion Prompting for Multilingual Reasoning on Multilingual Large Language Models [52.00446751692225]
  We present a novel and simple yet effective method called textbfDictionary textbfInsertion textbfPrompting (textbfDIP) When providing a non-English prompt, DIP looks up a word dictionary and inserts words' English counterparts into the prompt for LLMs. It then enables better translation into English and better English model thinking steps which leads to obviously better results.
  arXiv  Detail & Related papers  (2024-11-02T05:10:50Z)
• (Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Texts [52.18246881218829]
  We introduce a novel multi-agent framework based on large language models (LLMs) for literary translation, implemented as a company called TransAgents. To evaluate the effectiveness of our system, we propose two innovative evaluation strategies: Monolingual Human Preference (MHP) and Bilingual LLM Preference (BLP)
  arXiv  Detail & Related papers  (2024-05-20T05:55:08Z)
• Adapting Large Language Models for Document-Level Machine Translation [46.370862171452444]
  Large language models (LLMs) have significantly advanced various natural language processing (NLP) tasks. Recent research indicates that moderately-sized LLMs often outperform larger ones after task-specific fine-tuning. This study focuses on adapting LLMs for document-level machine translation (DocMT) for specific language pairs.
  arXiv  Detail & Related papers  (2024-01-12T09:29:13Z)
• Cross-Lingual Knowledge Editing in Large Language Models [73.12622532088564]
  Knowledge editing has been shown to adapt large language models to new knowledge without retraining from scratch. It is still unknown the effect of source language editing on a different target language. We first collect a large-scale cross-lingual synthetic dataset by translating ZsRE from English to Chinese.
  arXiv  Detail & Related papers  (2023-09-16T11:07:52Z)
• Exploring Human-Like Translation Strategy with Large Language Models [93.49333173279508]
  Large language models (LLMs) have demonstrated impressive capabilities in general scenarios. This work proposes the MAPS framework, which stands for Multi-Aspect Prompting and Selection. We employ a selection mechanism based on quality estimation to filter out noisy and unhelpful knowledge.
  arXiv  Detail & Related papers  (2023-05-06T19:03:12Z)
• Adaptive Machine Translation with Large Language Models [7.803471587734353]
  We investigate how we can utilize in-context learning to improve real-time adaptive machine translation. We conduct experiments across five diverse language pairs, namely English-to-Arabic (EN-AR), English-to-Chinese (EN-ZH), English-to-French (EN-FR), English-to-Kinyarwanda (EN-RW), and English-to-Spanish (EN-ES)
  arXiv  Detail & Related papers  (2023-01-30T21:17:15Z)
• Machine Translation for Accessible Multi-Language Text Analysis [1.5484595752241124]
  We show that English-trained measures computed after translation to English have adequate-to-excellent accuracy. We show this for three major analytics -- sentiment analysis, topic analysis, and word embeddings -- over 16 languages.
  arXiv  Detail & Related papers  (2023-01-20T04:11:38Z)
• Understanding Translationese in Cross-Lingual Summarization [106.69566000567598]
  Cross-lingual summarization (MS) aims at generating a concise summary in a different target language. To collect large-scale CLS data, existing datasets typically involve translation in their creation. In this paper, we first confirm that different approaches of constructing CLS datasets will lead to different degrees of translationese.
  arXiv  Detail & Related papers  (2022-12-14T13:41:49Z)
• Towards Debiasing Translation Artifacts [15.991970288297443]
  We propose a novel approach to reducing translationese by extending an established bias-removal technique. We use the Iterative Null-space Projection (INLP) algorithm, and show by measuring classification accuracy before and after debiasing, that translationese is reduced at both sentence and word level. To the best of our knowledge, this is the first study to debias translationese as represented in latent embedding space.
  arXiv  Detail & Related papers  (2022-05-16T21:46:51Z)
• How do lexical semantics affect translation? An empirical study [1.0152838128195467]
  A distinguishing factor of natural language is that words are typically ordered according to the rules of the grammar of a given language. We investigate how the word ordering of and lexical similarity between the source and target language affect translation performance.
  arXiv  Detail & Related papers  (2021-12-31T23:28:28Z)
• FILTER: An Enhanced Fusion Method for Cross-lingual Language Understanding [85.29270319872597]
  We propose an enhanced fusion method that takes cross-lingual data as input for XLM finetuning. During inference, the model makes predictions based on the text input in the target language and its translation in the source language. To tackle this issue, we propose an additional KL-divergence self-teaching loss for model training, based on auto-generated soft pseudo-labels for translated text in the target language.
  arXiv  Detail & Related papers  (2020-09-10T22:42:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.