MLQE-PE: A Multilingual Quality Estimation and Post-Editing Dataset

• URL: http://arxiv.org/abs/2010.04480v3
• Date: Mon, 11 Oct 2021 09:31:35 GMT
• Title: MLQE-PE: A Multilingual Quality Estimation and Post-Editing Dataset
• Abstract summary: The dataset contains eleven language pairs, with human labels for up to 10,000 translations per language pair. It also contains the post-edited sentences, as well as titles of the articles where the sentences were extracted from, and the neural MT models used to translate the text.
• Score: 49.602565530704005
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present MLQE-PE, a new dataset for Machine Translation (MT) Quality Estimation (QE) and Automatic Post-Editing (APE). The dataset contains eleven language pairs, with human labels for up to 10,000 translations per language pair in the following formats: sentence-level direct assessments and post-editing effort, and word-level good/bad labels. It also contains the post-edited sentences, as well as titles of the articles where the sentences were extracted from, and the neural MT models used to translate the text.

Related papers

• APEX-VW: A Document-Level English-Spanish Post-Editing Dataset in the Healthcare Domain [7.333648670424977]
  Post-Editing (PE) of Machine Translation (MT) output often involves repeating the same lexical and terminological corrections across many segments.<n>This paper presents the APEX-VW Corpus, a new open English-Spanish dataset built from recent NHS virtual-ward documents and professional translation in Trados Studio.<n>Unlike prior resources such as WMT APE corpora, eSCAPE, MLQE-PE, or LangMark, the dataset preserves document order and CAT-tool context.
  arXiv  Detail & Related papers  (2026-08-08T10:54:44Z)
• XQ-MEval: A Dataset with Cross-lingual Parallel Quality for Benchmarking Translation Metrics [64.77152900881724]
  We propose XQ-MEval, a semi-automatically built dataset covering nine translation directions.<n>We inject MQM-defined errors into gold translations automatically, filter them by native speakers for reliability, and merge errors to generate pseudo translations with controllable quality.<n>Using XQ-MEval, our experiments on nine representative metrics reveal the inconsistency between averaging and human judgment.
  arXiv  Detail & Related papers  (2026-04-16T12:27:10Z)
• Semi-Synthetic Parallel Data for Translation Quality Estimation: A Case Study of Dataset Building for an Under-Resourced Language Pair [0.0]
  This study presents a semi-synthetic parallel dataset for English-to-Hebrew QE.<n>We incorporated professionally translated English-Hebrew segments from our own resources, which were assigned the highest quality score.<n> Controlled translation errors were introduced to address linguistic challenges, particularly regarding gender and number agreement.
  arXiv  Detail & Related papers  (2026-03-12T09:48:34Z)
• Beyond Literal Mapping: Benchmarking and Improving Non-Literal Translation Evaluation [57.11989521509119]
  We propose a novel agentic translation evaluation framework, centered by a reflective Core Agent that invokes specialized sub-agents.<n> Experimental results indicate the efficacy of RATE, achieving an improvement of at least 3.2 meta score compared with current metrics.
  arXiv  Detail & Related papers  (2026-01-12T09:03:42Z)
• LangMark: A Multilingual Dataset for Automatic Post-Editing [13.873007459228987]
  LangMark is a new human-annotated multilingual APE dataset for English translation to seven languages.<n>The dataset has 206,983 triplets, with each triplet consisting of a source segment, its NMT output, and a human post-edited translation.
  arXiv  Detail & Related papers  (2025-11-21T11:18:15Z)
• HPLT 3.0: Very Large-Scale Multilingual Resources for LLM and MT. Mono- and Bi-lingual Data, Multilingual Evaluation, and Pre-Trained Models [25.953042884928006]
  We present an initiative to provide open, very large, high-quality, and richly annotated textual datasets for almost 200 languages.<n>At 30 trillion tokens, this is likely the largest generally available multilingual collection of LLM pre-training data.<n>We train and evaluate a family of 57 monolingual encoder-decoder models, as well as a handful of monolingual GPT-like reference models.
  arXiv  Detail & Related papers  (2025-11-02T20:16:38Z)
• Alleviating Distribution Shift in Synthetic Data for Machine Translation Quality Estimation [55.73341401764367]
  We introduce ADSQE, a novel framework for alleviating distribution shift in synthetic QE data. ADSQE uses references, i.e., translation supervision signals, to guide both the generation and annotation processes. Experiments demonstrate that ADSQE outperforms SOTA baselines like COMET in both supervised and unsupervised settings.
  arXiv  Detail & Related papers  (2025-02-27T10:11:53Z)
• A Data Selection Approach for Enhancing Low Resource Machine Translation Using Cross-Lingual Sentence Representations [0.4499833362998489]
  This study focuses on the case of English-Marathi language pairs, where existing datasets are notably noisy. To mitigate the impact of data quality issues, we propose a data filtering approach based on cross-lingual sentence representations. Results demonstrate a significant improvement in translation quality over the baseline post-filtering with IndicSBERT.
  arXiv  Detail & Related papers  (2024-09-04T13:49:45Z)
• LexMatcher: Dictionary-centric Data Collection for LLM-based Machine Translation [67.24113079928668]
  We present LexMatcher, a method for data curation driven by the coverage of senses found in bilingual dictionaries. Our approach outperforms the established baselines on the WMT2022 test sets.
  arXiv  Detail & Related papers  (2024-06-03T15:30:36Z)
• FAME-MT Dataset: Formality Awareness Made Easy for Machine Translation Purposes [0.0]
  We present FAME-MT -- a dataset consisting of 11.2 million translations between 15 European source languages and 8 European target languages. This dataset can be used to fine-tune machine translation models to ensure a given formality level for each European target language. We present a publicly available proof-of-concept machine translation model that uses the dataset to steer the formality level of the translation.
  arXiv  Detail & Related papers  (2024-05-20T10:35:30Z)
• Statistical Machine Translation for Indic Languages [1.8899300124593648]
  This paper canvasses about the development of bilingual Statistical Machine Translation models. To create the system, MOSES open-source SMT toolkit is explored. In our experiment, the quality of the translation is evaluated using standard metrics such as BLEU, METEOR, and RIBES.
  arXiv  Detail & Related papers  (2023-01-02T06:23:12Z)
• FRMT: A Benchmark for Few-Shot Region-Aware Machine Translation [64.9546787488337]
  We present FRMT, a new dataset and evaluation benchmark for Few-shot Region-aware Machine Translation. The dataset consists of professional translations from English into two regional variants each of Portuguese and Mandarin Chinese.
  arXiv  Detail & Related papers  (2022-10-01T05:02:04Z)
• Rethink about the Word-level Quality Estimation for Machine Translation from Human Judgement [57.72846454929923]
  We create a benchmark dataset, emphHJQE, where the expert translators directly annotate poorly translated words. We propose two tag correcting strategies, namely tag refinement strategy and tree-based annotation strategy, to make the TER-based artificial QE corpus closer to emphHJQE. The results show our proposed dataset is more consistent with human judgement and also confirm the effectiveness of the proposed tag correcting strategies.
  arXiv  Detail & Related papers  (2022-09-13T02:37:12Z)
• DivEMT: Neural Machine Translation Post-Editing Effort Across Typologically Diverse Languages [5.367993194110256]
  DivEMT is the first publicly available post-editing study of Neural Machine Translation (NMT) over a typologically diverse set of target languages. We assess the impact on translation productivity of two state-of-the-art NMT systems, namely: Google Translate and the open-source multilingual model mBART50.
  arXiv  Detail & Related papers  (2022-05-24T17:22:52Z)
• Neural Label Search for Zero-Shot Multi-Lingual Extractive Summarization [80.94424037751243]
  In zero-shot multilingual extractive text summarization, a model is typically trained on English dataset and then applied on summarization datasets of other languages. We propose NLS (Neural Label Search for Summarization), which jointly learns hierarchical weights for different sets of labels together with our summarization model. We conduct multilingual zero-shot summarization experiments on MLSUM and WikiLingua datasets, and we achieve state-of-the-art results using both human and automatic evaluations.
  arXiv  Detail & Related papers  (2022-04-28T14:02:16Z)
• NLPDove at SemEval-2020 Task 12: Improving Offensive Language Detection with Cross-lingual Transfer [10.007363787391952]
  This paper describes our approach to the task of identifying offensive languages in a multilingual setting. We investigate two data augmentation strategies: using additional semi-supervised labels with different thresholds and cross-lingual transfer with data selection. Our multilingual systems achieved competitive results in Greek, Danish, and Turkish at OffensEval 2020.
  arXiv  Detail & Related papers  (2020-08-04T06:20:50Z)
• A High-Quality Multilingual Dataset for Structured Documentation Translation [101.41835967142521]
  This paper presents a high-quality multilingual dataset for the documentation domain. We collect XML-structured parallel text segments from the online documentation for an enterprise software platform.
  arXiv  Detail & Related papers  (2020-06-24T02:08:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.