Shared Doubt: Zero-shot Cross-Lingual Confidence Estimation for Language Models

• URL: http://arxiv.org/abs/2605.31220v1
• Date: Fri, 29 May 2026 12:25:24 GMT
• Title: Shared Doubt: Zero-shot Cross-Lingual Confidence Estimation for Language Models
• Abstract summary: We use a lightweight linear probe that predicts answer correctness directly from intermediate representations.<n> Learned layer weights and multiple ablations reveal that confidence features concentrate in middle layers across languages.<n>While zero-shot cross-lingual performance depends on similarity to the source language, the probe provides a strong baseline without any retraining.
• Score: 14.815594636224747
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Confidence estimation (CE), i.e. quantifying the reliability of a model's prediction, has attracted great interest in the context of large language models (LLMs). However, most studies focus on English, ignoring the multilingual reality of LLM usage, while many CE methods degrade or require retraining across languages. To address this gap, we investigate whether multilingual LLMs encode shared, language-transferable confidence features. We use a lightweight linear probe that predicts answer correctness directly from intermediate representations. Trained monolingually, the probe generalizes zero-shot to unseen, typologically diverse languages without target-language supervision. Learned layer weights and multiple ablations reveal that confidence features concentrate in middle layers across languages, suggesting a shared confidence subspace. While zero-shot cross-lingual performance depends on similarity to the source language, the probe provides a strong baseline without any retraining and compares favorably to other popular confidence estimation methods.

Related papers

• Cross-lingual Representation Learning via Centroid Intervention Fusion [30.50994950370956]
  Centroid Intervention Fusion (CIF) is a projection fusion framework that consolidates multiple multilingual intervention projections into a single language-shared operator.<n>CIF outperforms the strongest prior pairwise intervention baseline by up to +3.378 pp on average across four model backbones.
  arXiv  Detail & Related papers  (2026-08-26T19:39:57Z)
• Inference-Time Steering for Cross-Lingual Factual Consistency in LLMs [51.56484100374058]
  Large Language Models (LLMs) demonstrate remarkable multilingual fluency, but their internal knowledge representations remain disproportionately biased toward high-resource languages.<n>We investigate whether these biases can be mitigated at inference time, forcing an English-prompted model to answer as if it were queried in target languages.<n>Experiments on Gemma 3 12B Instruct reveal persona prompting to be the strongest overall intervention, balancing efficacy, safety, and out-of-domain generalization.
  arXiv  Detail & Related papers  (2026-07-21T16:15:05Z)
• Estimating Uncertainty from Reasoning: A Large-Scale Study of Multi- and Crosslingual MCQA Performance in LLMs [15.831021059811045]
  Uncertainty estimation (UE) enables LLM-powered systems to recognize when to abstain.<n>We present the first large-scale evaluation of UE methods across 22 languages.
  arXiv  Detail & Related papers  (2026-07-07T14:25:09Z)
• Optimizing Language Models for Crosslingual Knowledge Consistency [90.86445137816942]
  Large language models are known to often exhibit inconsistent knowledge.<n>This is particularly problematic in multilingual scenarios, where models are likely to be asked similar questions in different languages.<n>In this work, we show that this issue can be mitigated using reinforcement learning with a structured reward function.
  arXiv  Detail & Related papers  (2026-03-04T23:36:55Z)
• Investigating the Multilingual Calibration Effects of Language Model Instruction-Tuning [58.355275813623685]
  This work looks at a critical gap in the calibration of large language models (LLMs) within multilingual settings.<n>Even in low-resource languages, model confidence can increase significantly after instruction-tuning on high-resource language SFT datasets.<n>However, improvements in accuracy are marginal or non-existent, highlighting a critical shortcoming of standard SFT for multilingual languages.
  arXiv  Detail & Related papers  (2026-01-04T04:29:12Z)
• Beyond the Final Layer: Intermediate Representations for Better Multilingual Calibration in Large Language Models [50.34755385896279]
  Confidence calibration is crucial for the reliable deployment of Large Language Models (LLMs)<n>We conduct the first large-scale, systematic studies of multilingual calibration across six model families and over 100 languages.<n>We find that non-English languages suffer from systematically worse calibration.
  arXiv  Detail & Related papers  (2025-10-03T16:07:15Z)
• Can Large Language Models Express Uncertainty Like Human? [71.27418419522884]
  We release the first diverse, large-scale dataset of hedging expressions with human-annotated confidence scores.<n>We conduct the first systematic study of linguistic confidence across modern large language models.
  arXiv  Detail & Related papers  (2025-09-29T02:34:30Z)
• Cross-Lingual Pitfalls: Automatic Probing Cross-Lingual Weakness of Multilingual Large Language Models [55.14276067678253]
  This paper introduces a novel methodology for efficiently identifying inherent cross-lingual weaknesses in Large Language Models (LLMs)<n>We construct a new dataset of over 6,000 bilingual pairs across 16 languages using this methodology, demonstrating its effectiveness in revealing weaknesses even in state-of-the-art models.<n>Further experiments investigate the relationship between linguistic similarity and cross-lingual weaknesses, revealing that linguistically related languages share similar performance patterns.
  arXiv  Detail & Related papers  (2025-05-24T12:31:27Z)
• Scaling Test-time Compute for Low-resource Languages: Multilingual Reasoning in LLMs [3.9530780161144667]
  We investigate the multilingual mechanism by which Large Language Models internally operate in a latent space biased toward their inherently dominant language.<n>We train models to generate the chain-of-thought (CoT) in English while outputting the final response in the target language, given input in the low-resource language.<n>Our experiments demonstrate that this approach, named English-Pivoted CoT Training, outperforms other baselines, with up to 28.33% improvement.
  arXiv  Detail & Related papers  (2025-04-02T16:58:36Z)
• MlingConf: A Comprehensive Study of Multilingual Confidence Estimation on Large Language Models [36.33453112932689]
  This paper introduces a comprehensive investigation of Multilingual Confidence estimation (MlingConf) on Large Language Models (LLMs)<n>The benchmark comprises four meticulously checked and human-evaluated high-quality multilingual datasets for LA tasks and one for the LS task tailored to specific social, cultural, and geographical contexts of a language.<n>Experiments reveal that on LA tasks English exhibits notable linguistic dominance in confidence estimations than other languages, while on LS tasks, using question-related language to prompt LLMs demonstrates better linguistic dominance in multilingual confidence estimations.
  arXiv  Detail & Related papers  (2024-02-21T08:20:06Z)
• From Good to Best: Two-Stage Training for Cross-lingual Machine Reading Comprehension [51.953428342923885]
  We develop a two-stage approach to enhance the model performance. The first stage targets at recall: we design a hard-learning (HL) algorithm to maximize the likelihood that the top-k predictions contain the accurate answer. The second stage focuses on precision: an answer-aware contrastive learning mechanism is developed to learn the fine difference between the accurate answer and other candidates.
  arXiv  Detail & Related papers  (2021-12-09T07:31:15Z)
• Boosting Cross-Lingual Transfer via Self-Learning with Uncertainty Estimation [34.97086123805344]
  Recent multilingual pre-trained language models have achieved remarkable zero-shot performance. We propose a self-learning framework that further utilizes unlabeled data of target languages. We evaluate our framework with uncertainties on two cross-lingual tasks including Named Entity Recognition (NER) and Natural Language Inference (NLI) covering 40 languages in total.
  arXiv  Detail & Related papers  (2021-09-01T05:26:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.