Domain-Specific Quality Estimation for Machine Translation in Low-Resource Scenarios

• URL: http://arxiv.org/abs/2603.07372v1
• Date: Sat, 07 Mar 2026 22:55:25 GMT
• Title: Domain-Specific Quality Estimation for Machine Translation in Low-Resource Scenarios
• Abstract summary: We investigate sentence-level QE for English to Indic machine translation across four domains (Healthcare, Legal, Tourism, and General) and five language pairs.<n>Findings indicate that while closed-weight models achieve strong performance via prompting alone, prompt-only approaches remain fragile for open-weight models.
• Score: 10.17317882426833
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Quality Estimation (QE) is essential for assessing machine translation quality in reference-less settings, particularly for domain-specific and low-resource language scenarios. In this paper, we investigate sentence-level QE for English to Indic machine translation across four domains (Healthcare, Legal, Tourism, and General) and five language pairs. We systematically compare zero-shot, few-shot, and guideline-anchored prompting across selected closed-weight and open-weight LLMs. Findings indicate that while closed-weight models achieve strong performance via prompting alone, prompt-only approaches remain fragile for open-weight models, especially in high-risk domains. To address this, we adopt ALOPE, a framework for LLM-based QE that uses Low-Rank Adaptation with regression heads attached to selected intermediate Transformer layers. We also extend ALOPE with recently proposed Low-Rank Multiplicative Adaptation (LoRMA). Our results show that intermediate-layer adaptation consistently improves QE performance, with gains in semantically complex domains, indicating a path toward more robust QE in practical scenarios. We release code and domain-specific QE datasets publicly to support further research.

Related papers

• Toward domain-specific machine translation and quality estimation systems [0.0]
  Machine Translation (MT) and Quality Estimation (QE) perform well in general domains but degrade under domain mismatch.<n>This dissertation studies how to adapt MT and QE systems to specialized domains through a set of data-focused contributions.
  arXiv  Detail & Related papers  (2026-03-26T02:47:40Z)
• Evidence-based Distributional Alignment for Large Language Models [58.65469623911573]
  We propose Evi-DA, an evidence-based alignment technique that improves the fidelity and robustness of LLM-based distribution estimation.<n>Given a target country, Evi-DA retrieves related World Values Survey items and their answer distributions, predicts a coarse Welzel value signature for each option, and infers the country-conditioned answer distribution in a structured format.
  arXiv  Detail & Related papers  (2026-03-03T03:34:06Z)
• Beyond Scalar Scores: Reinforcement Learning for Error-Aware Quality Estimation of Machine Translation [10.050982803590903]
  Quality Estimation aims to assess the quality of machine translation (MT) outputs without relying on reference translations.<n>We introduce the first segment-level QE dataset for English to Malayalam, a severely resource-scarce language pair.<n>We introduce ALOPE-RL, a policy-based reinforcement learning framework that trains efficient adapters.
  arXiv  Detail & Related papers  (2026-02-09T12:42:41Z)
• Beyond Many-Shot Translation: Scaling In-Context Demonstrations For Low-Resource Machine Translation [49.82863380286994]
  In-context learning may offer novel ways to adapt Large Language Models for low-resource machine translation.<n>In this study, we explore scaling low-resource machine translation ICL beyond the few-shot setting to thousands of examples with long-context models.<n>Our experiments on Javanese and Sundanese show that gains from additional context saturate quickly and can degrade near the maximum context window.
  arXiv  Detail & Related papers  (2026-02-04T17:02:22Z)
• ALOPE: Adaptive Layer Optimization for Translation Quality Estimation using Large Language Models [8.518082319087375]
  Large Language Models (LLMs) have shown remarkable performance across a wide range of natural language processing tasks.<n>QE for Machine Translation (MT) assesses the quality of a source-target pair without relying on reference translations.<n>This paper introduces ALOPE, an adaptive layer-optimization framework designed to enhance LLM-based QE.
  arXiv  Detail & Related papers  (2025-08-10T20:59:44Z)
• Alleviating Distribution Shift in Synthetic Data for Machine Translation Quality Estimation [55.73341401764367]
  We introduce DCSQE, a novel framework for alleviating distribution shift in synthetic QE data.<n> DCSQE uses references, i.e., translation supervision signals, to guide both the generation and annotation processes.<n>Experiments demonstrate that DCSQE outperforms SOTA baselines in both supervised and unsupervised settings.
  arXiv  Detail & Related papers  (2025-02-27T10:11:53Z)
• When LLMs Struggle: Reference-less Translation Evaluation for Low-resource Languages [9.138590152838754]
  Segment-level quality estimation (QE) is a challenging cross-lingual language understanding task.<n>We comprehensively evaluate large language models (LLMs) in zero/few-shot scenarios.<n>Our results indicate that prompt-based approaches are outperformed by the encoder-based fine-tuned QE models.
  arXiv  Detail & Related papers  (2025-01-08T12:54:05Z)
• Exploring Language Model Generalization in Low-Resource Extractive QA [57.14068405860034]
  We investigate Extractive Question Answering (EQA) with Large Language Models (LLMs) under domain drift.<n>We devise a series of experiments to explain the performance gap empirically.
  arXiv  Detail & Related papers  (2024-09-27T05:06:43Z)
• Don't Rank, Combine! Combining Machine Translation Hypotheses Using Quality Estimation [0.6998085564793366]
  This work introduces QE-fusion, a method that synthesizes translations using a quality estimation metric (QE) We demonstrate that our approach generates novel translations in over half of the cases. We empirically establish that QE-fusion scales linearly with the number of candidates in the pool.
  arXiv  Detail & Related papers  (2024-01-12T16:52:41Z)
• QAmeleon: Multilingual QA with Only 5 Examples [71.80611036543633]
  We show how to leverage pre-trained language models under a few-shot learning setting. Our approach, QAmeleon, uses a PLM to automatically generate multilingual data upon which QA models are trained. Prompt tuning the PLM for data synthesis with only five examples per language delivers accuracy superior to translation-based baselines.
  arXiv  Detail & Related papers  (2022-11-15T16:14:39Z)
• Non-Parametric Unsupervised Domain Adaptation for Neural Machine Translation [61.27321597981737]
  $k$NN-MT has shown the promising capability of directly incorporating the pre-trained neural machine translation (NMT) model with domain-specific token-level $k$-nearest-neighbor retrieval. We propose a novel framework that directly uses in-domain monolingual sentences in the target language to construct an effective datastore for $k$-nearest-neighbor retrieval.
  arXiv  Detail & Related papers  (2021-09-14T11:50:01Z)
• Knowledge Distillation for Quality Estimation [79.51452598302934]
  Quality Estimation (QE) is the task of automatically predicting Machine Translation quality in the absence of reference translations. Recent success in QE stems from the use of multilingual pre-trained representations, where very large models lead to impressive results. We show that this approach, in combination with data augmentation, leads to light-weight QE models that perform competitively with distilled pre-trained representations with 8x fewer parameters.
  arXiv  Detail & Related papers  (2021-07-01T12:36:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.