The Translation Tax Is Not a Scalar: A Counterfactual Audit of English-Source Cue Inheritance in Chinese Multilingual Benchmarks

• URL: http://arxiv.org/abs/2605.07093v1
• Date: Fri, 08 May 2026 01:18:46 GMT
• Title: The Translation Tax Is Not a Scalar: A Counterfactual Audit of English-Source Cue Inheritance in Chinese Multilingual Benchmarks
• Abstract summary: The Translation Tax is often treated as a scalar: translated benchmarks are assumed to inflate scores by preserving English-source cues.<n>Three proxy estimators disagree: back-translation gaps are small and-fragile; cue-score calibration does not predict item-level gains.<n>We release per-cell evidence, the naturalization protocol, human QC, and a reporting checklist for translated multilingual benchmark papers.
• Score: 1.4839450468199458
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The Translation Tax is often treated as a scalar: translated benchmarks are assumed to inflate scores by preserving English-source cues. We audit this claim in an English-to-Chinese setting. Three proxy estimators disagree: back-translation gaps are small and parser-fragile; cue-score calibration does not predict item-level gains; and a six-model native-control comparison shows model-family rather than uniform benchmark effects. We add a same-item LLM-naturalization stress test that holds answer, options, and content fixed while rewriting Chinese surface form. After correcting a prompt-construction bug, this contrast no longer supports a model-family interaction, but it preserves a residue dose-response: high-residue items benefit while low-residue items do not. The result is not a single Translation Tax, but a set of estimator- and item-dependent validity risks. We release per-cell evidence, the naturalization protocol, human QC, and a reporting checklist for translated multilingual benchmark papers.

Related papers

• Language-Specific Gaps in AI Safety Training Datasets [0.0]
  Self-harm and sexual-content categories have no native-language coverage in either African-language tier we studied.<n>We show that self-harm and sexual-content categories have no native-language coverage in either African-language tier we studied.
  arXiv  Detail & Related papers  (2026-08-13T18:40:22Z)
• Contrastive ESA: Human Evaluation of Multiple Translations at Once [45.918020796451536]
  We introduce Contrastive Error Span.<n>(cESA) a protocol that presents multiple translations of the source input (text, video, audio, image).<n>In cESA, the annotator sees multiple translations of the same document, marks major and minor error spans, and then assigns a score from 0% to 100% on absolute scale.<n>We validate cESA using a large-scale human evaluation of English->Japanese translations of 12 models, demonstrating reductions in annotation time and noise compared to standard pointwise evaluation.
  arXiv  Detail & Related papers  (2026-07-29T09:01:59Z)
• Inference-Time Steering for Cross-Lingual Factual Consistency in LLMs [51.56484100374058]
  Large Language Models (LLMs) demonstrate remarkable multilingual fluency, but their internal knowledge representations remain disproportionately biased toward high-resource languages.<n>We investigate whether these biases can be mitigated at inference time, forcing an English-prompted model to answer as if it were queried in target languages.<n>Experiments on Gemma 3 12B Instruct reveal persona prompting to be the strongest overall intervention, balancing efficacy, safety, and out-of-domain generalization.
  arXiv  Detail & Related papers  (2026-07-21T16:15:05Z)
• Lower-Resource, Higher Scores: Language Bias in LLM Evaluators [27.959960538188753]
  We conduct experiments with semantically identical instruction-response pairs across 23 languages.<n>We find that multilingual evaluators assign significantly different scores to different evaluation languages.
  arXiv  Detail & Related papers  (2026-07-16T01:53:09Z)
• A Komi-Yazva--Russian Parallel Corpus and Evaluation Protocol for Zero- and Few-Shot LLM Translation [0.0]
  We present the first Komi-Yazva--Russian parallel corpus together with an explicit evaluation protocol.<n>The dataset contains 457 aligned sentence pairs from 74 narrative texts.<n>We use this setup to compare modern large language models on Komi-Yazva-to-Russian translation under severe parallel-data scarcity.
  arXiv  Detail & Related papers  (2026-06-04T17:26:38Z)
• Quantifying the Impact of Translation Errors on Multilingual LLM Evaluation [8.48956061615262]
  Machine-translated benchmarks are widely used to assess the multilingual capabilities of large language models (LLMs)<n>We address two practical gaps: how well automatic MQM-style error spans from LLM judges and a span-aware QE baseline.<n>We find that span agreement is non-trivial on naturally occurring benchmark translations, and that target-side translation errors are consistently associated with measurable, percentage-point drops in translated accuracy.
  arXiv  Detail & Related papers  (2026-05-24T07:06:34Z)
• XQ-MEval: A Dataset with Cross-lingual Parallel Quality for Benchmarking Translation Metrics [64.77152900881724]
  We propose XQ-MEval, a semi-automatically built dataset covering nine translation directions.<n>We inject MQM-defined errors into gold translations automatically, filter them by native speakers for reliability, and merge errors to generate pseudo translations with controllable quality.<n>Using XQ-MEval, our experiments on nine representative metrics reveal the inconsistency between averaging and human judgment.
  arXiv  Detail & Related papers  (2026-04-16T12:27:10Z)
• Plausibility as Commonsense Reasoning: Humans Succeed, Large Language Models Do not [0.0]
  We test whether large language models integrate world knowledge with syntactic structure in a human-like, structure-sensitive way during ambiguity resolution.<n>We construct ambiguous items that keep the syntactic configuration fixed and ensure both parses remain pragmatically possible.<n>In a speeded forced-choice comprehension experiment, humans show a large, correctly directed plausibility effect.<n>Across models, plausibility-driven shifts are weak, unstable, or reversed.
  arXiv  Detail & Related papers  (2026-04-06T16:27:06Z)
• Mitigating Translationese Bias in Multilingual LLM-as-a-Judge via Disentangled Information Bottleneck [37.780081880731096]
  Large language models (LLMs) have become a standard for multilingual evaluation, yet they exhibit a severe systematic translationese bias.<n>We attribute this bias to spurious correlations with (i) latent manifold alignment with English and (ii) cross-lingual predictability.<n>To mitigate this bias, we propose DIBJudge, a robust fine-tuning framework that learns a minimally sufficient, judgment-critical representation.
  arXiv  Detail & Related papers  (2026-03-11T02:55:29Z)
• CROP: Zero-shot Cross-lingual Named Entity Recognition with Multilingual Labeled Sequence Translation [113.99145386490639]
  Cross-lingual NER can transfer knowledge between languages via aligned cross-lingual representations or machine translation results. We propose a Cross-lingual Entity Projection framework (CROP) to enable zero-shot cross-lingual NER. We adopt a multilingual labeled sequence translation model to project the tagged sequence back to the target language and label the target raw sentence.
  arXiv  Detail & Related papers  (2022-10-13T13:32:36Z)
• Rethinking Round-Trip Translation for Machine Translation Evaluation [44.83568796515321]
  We report the surprising finding that round-trip translation can be used for automatic evaluation without the references. We demonstrate the rectification is overdue as round-trip translation could benefit multiple machine translation evaluation tasks.
  arXiv  Detail & Related papers  (2022-09-15T15:06:20Z)
• Mismatching-Aware Unsupervised Translation Quality Estimation For Low-Resource Languages [6.049660810617423]
  XLMRScore is a cross-lingual counterpart of BERTScore computed via the XLM-RoBERTa (XLMR) model. We evaluate the proposed method on four low-resource language pairs of the WMT21 QE shared task.
  arXiv  Detail & Related papers  (2022-07-31T16:23:23Z)
• ChrEnTranslate: Cherokee-English Machine Translation Demo with Quality Estimation and Corrective Feedback [70.5469946314539]
  ChrEnTranslate is an online machine translation demonstration system for translation between English and an endangered language Cherokee. It supports both statistical and neural translation models as well as provides quality estimation to inform users of reliability.
  arXiv  Detail & Related papers  (2021-07-30T17:58:54Z)
• On the Limitations of Cross-lingual Encoders as Exposed by Reference-Free Machine Translation Evaluation [55.02832094101173]
  Evaluation of cross-lingual encoders is usually performed either via zero-shot cross-lingual transfer in supervised downstream tasks or via unsupervised cross-lingual similarity. This paper concerns ourselves with reference-free machine translation (MT) evaluation where we directly compare source texts to (sometimes low-quality) system translations. We systematically investigate a range of metrics based on state-of-the-art cross-lingual semantic representations obtained with pretrained M-BERT and LASER. We find that they perform poorly as semantic encoders for reference-free MT evaluation and identify their two key limitations.
  arXiv  Detail & Related papers  (2020-05-03T22:10:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.