ComplexityMT: Benchmarking the Interaction Between Text Complexity and Machine Translation

• URL: http://arxiv.org/abs/2606.05421v1
• Date: Wed, 03 Jun 2026 20:38:37 GMT
• Title: ComplexityMT: Benchmarking the Interaction Between Text Complexity and Machine Translation
• Abstract summary: We introduce ComplexityMT, a new challenge for assessing how text complexity and machine translation interact with and influence each other.<n>We evaluate three open-weight models, one closed model, and a commercial machine translation system on two tasks.<n>Our experiments show that higher CEFR levels make texts more difficult to translate, and that machine translation shifts the CEFR level of the target text compared to the original source.
• Score: 48.0972692373962
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: When a text is translated, does the translation retain the complexity of the original? We introduce ComplexityMT, a new challenge for assessing how text complexity and machine translation interact with and influence each other, using the Common European Framework of Reference for Languages (CEFR) levels as the measure of text complexity. Across six languages, including Arabic, Dutch, English, French, Hindi, and Russian, we evaluate three open-weight models, one closed model, and a commercial machine translation system on two tasks: i) correlation of CEFR with translation difficulty, and ii) shifts in CEFR levels of the source texts. Our experiments show that higher CEFR levels make texts more difficult to translate, and that machine translation shifts the CEFR level of the target text compared to the original source, for most languages. These findings provide new insights for researchers and practitioners working on multilingual pedagogical content generation and machine translation difficulty estimation.

Related papers

• Translationese as a Rational Response to Translation Task Difficulty [0.0]
  We test whether observable translationese can be predicted from quantifiable measures of translation task difficulty.<n>We use a bidirectional English-German corpus comprising written and spoken subcorpora.<n>Results indicate that translationese can be partly explained by translation task difficulty, especially in English-to-German.
  arXiv  Detail & Related papers  (2026-03-12T15:24:00Z)
• Liaozhai through the Looking-Glass: On Paratextual Explicitation of Culture-Bound Terms in Machine Translation [70.43884512651668]
  We formalize Genette's (1987) theory of paratexts from literary and translation studies to introduce the task of paratextual explicitation for machine translation.<n>We construct a dataset of 560 expert-aligned paratexts from four English translations of the classical Chinese short story collection Liaozhai.<n>Our findings demonstrate the potential of paratextual explicitation in advancing machine translation beyond linguistic equivalence.
  arXiv  Detail & Related papers  (2025-09-27T16:27:36Z)
• Estimating Machine Translation Difficulty [48.659971048116766]
  We formalize the task of translation difficulty estimation, defining a text's difficulty based on the expected quality of its translations.<n>We demonstrate the practical utility of difficulty estimators by using them to construct more challenging benchmarks for machine translation.<n>We release two improved models for difficulty estimation, Sentinel-src-24 and Sentinel-src-25.
  arXiv  Detail & Related papers  (2025-08-13T20:22:58Z)
• Towards Cross-Cultural Machine Translation with Retrieval-Augmented Generation from Multilingual Knowledge Graphs [18.84670051328337]
  XC-Translate is the first large-scale, manually-created benchmark for machine translation. KG-MT is a novel end-to-end method to integrate information from a multilingual knowledge graph into a neural machine translation model.
  arXiv  Detail & Related papers  (2024-10-17T21:56:22Z)
• LLM-based Translation Inference with Iterative Bilingual Understanding [52.46978502902928]
  We propose a novel Iterative Bilingual Understanding Translation method based on the cross-lingual capabilities of large language models (LLMs)<n>The cross-lingual capability of LLMs enables the generation of contextual understanding for both the source and target languages separately.<n>The proposed IBUT outperforms several strong comparison methods.
  arXiv  Detail & Related papers  (2024-10-16T13:21:46Z)
• (Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Texts [56.7988577327046]
  We introduce TransAgents, a novel multi-agent framework that simulates the roles and collaborative practices of a human translation company.<n>Our findings highlight the potential of multi-agent collaboration in enhancing translation quality, particularly for longer texts.
  arXiv  Detail & Related papers  (2024-05-20T05:55:08Z)
• Discourse Centric Evaluation of Machine Translation with a Densely Annotated Parallel Corpus [82.07304301996562]
  This paper presents a new dataset with rich discourse annotations, built upon the large-scale parallel corpus BWB introduced in Jiang et al. We investigate the similarities and differences between the discourse structures of source and target languages. We discover that MT outputs differ fundamentally from human translations in terms of their latent discourse structures.
  arXiv  Detail & Related papers  (2023-05-18T17:36:41Z)
• DivEMT: Neural Machine Translation Post-Editing Effort Across Typologically Diverse Languages [5.367993194110256]
  DivEMT is the first publicly available post-editing study of Neural Machine Translation (NMT) over a typologically diverse set of target languages. We assess the impact on translation productivity of two state-of-the-art NMT systems, namely: Google Translate and the open-source multilingual model mBART50.
  arXiv  Detail & Related papers  (2022-05-24T17:22:52Z)
• mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs [51.67970832510462]
  We improve multilingual text-to-text transfer Transformer with translation pairs (mT6) We explore three cross-lingual text-to-text pre-training tasks, namely, machine translation, translation pair span corruption, and translation span corruption. Experimental results show that the proposed mT6 improves cross-lingual transferability over mT5.
  arXiv  Detail & Related papers  (2021-04-18T03:24:07Z)
• Towards Personalised and Document-level Machine Translation of Dialogue [0.0]
  This thesis proposal focuses on PersNMT and DocNMT for the domain of dialogue extracted from TV subtitles in five languages. Three main challenges are addressed: (1) incorporating extra-textual information directly into NMT systems; (2) improving the machine translation of cohesion devices; and (3) reliable evaluation for PersNMT and DocNMT.
  arXiv  Detail & Related papers  (2021-02-11T09:18:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.