What Single-Prompt Accuracy Misses: A Multi-Variant Reliability Audit of Language Models

• URL: http://arxiv.org/abs/2605.02038v1
• Date: Sun, 03 May 2026 20:05:08 GMT
• Title: What Single-Prompt Accuracy Misses: A Multi-Variant Reliability Audit of Language Models
• Abstract summary: Single-prompt accuracy is the dominant way to benchmark language models, but it can miss reliability failures that matter.<n>We evaluate a 15-model open-weight corpus, with the main reliability analyses focused on 10 instruct models across five classification and reasoning benchmarks.<n>We find three broad results. First, evaluation design can materially change the conclusion.<n>Second, confidence signals are fragile. On MMLU-Pro, every primary model verbally reports confidence substantially above both its accuracy and its token-probability confidence on the same rows, and verbal parse rate can collapse for a single model on a single prompt variant.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Single-prompt accuracy is the dominant way to benchmark language models, but it can miss reliability failures that matter. We evaluate a 15-model open-weight corpus, with the main reliability analyses focused on 10 instruct models across five classification and reasoning benchmarks under five prompt variants each, measuring accuracy, token-probability calibration, verbal-confidence calibration, verbal parse rate, and prompt-perturbation spread for every (model x dataset x variant) cell. We find three broad results. First, evaluation design can materially change the conclusion. Switching Expected Calibration Error (ECE) token from a raw to a label-set-normalised definition changes per-cell calibration by a mean absolute 0.149. More strikingly, pairing a chain-of-thought prompt with a first-character evaluator on ARC-Challenge reduces apparent accuracy by 72-88% across all five primary models; two independent repair procedures recover 93.8% and 102.7% of the lost performance, indicating an evaluator-side rather than model-side failure. Second, confidence signals are fragile. On MMLU-Pro, every primary model verbally reports confidence substantially above both its accuracy and its token-probability confidence on the same rows, and verbal parse rate can collapse for a single model on a single prompt variant. Third, prompt robustness does not track parameter count reliably. Across 10 instruct models, the correlation between model size and prompt-perturbation spread ranges from -0.244 to 0.474 across benchmarks. Taken together, these results show that reliability conclusions for small language models depend not only on the model being evaluated, but also on the evaluation pipeline used to measure it. We argue that calibration definitions, evaluator logic, verbal parseability, and prompt robustness should be reported explicitly when making reliability claims.

Related papers

• Calibration-Aware Uncertainty Cascades for Efficient Heterogeneous Model Collaboration [18.045450334322545]
  Heterogeneous model collaboration seeks to exploit the complementary strengths of different models to balance predictive performance and inference cost.<n>Existing approaches typically rely on trained routers, which tie routing decisions to a fixed task and model pool, or on raw-confidence cascades, whose thresholds lack consistent reliability semantics across heterogeneous models.<n>We propose-Aware Uncertainty Cascades (CAUC), a simple post-hoc framework that independently calibrates each model's confidence and selects deployment policies using validation data.
  arXiv  Detail & Related papers  (2026-09-10T12:14:54Z)
• ConfidenceBench: Evaluating Confidence Calibration in Large Language Models [14.831793255747181]
  ConfidenceBench is a calibration benchmark for large language models.<n>It evaluates verbalized confidence estimates in 15 frontier large language models.<n> Confidence is elicited via prompting, requiring no access to model logits.
  arXiv  Detail & Related papers  (2026-07-10T01:56:30Z)
• CaliDist: Calibrating Large Language Models via Behavioral Robustness to Distraction [51.56484100374058]
  Existing calibration methods for Large Language Models (LLMs) often overlook a critical dimension of trustworthiness: a model's behavioral robustness.<n>We introduce textscCaliDist, a novel post-hoc calibration approach that directly measures and penalizes a model's susceptibility to distraction.<n>textscCaliDist consistently achieves lower Expected Error (ECE) and Brier Score compared with strong baselines.
  arXiv  Detail & Related papers  (2026-06-04T07:27:53Z)
• Calibration vs Decision Making: Revisiting the Reliability Paradox in Unlearned Language Models [15.63775421273161]
  We investigate the gap in generative language models using the multiple-choice question-answering evaluation protocol on the TOFU benchmark.<n>We find that fine-tuned models achieve low calibration error (ECE 0.04) compared to pretrained models.<n>Models after unlearning retain similarly low calibration despite reduced accuracy on the forget split.
  arXiv  Detail & Related papers  (2026-05-20T08:59:23Z)
• Verbal Confidence Saturation in 3-9B Open-Weight Instruction-Tuned LLMs: A Pre-Registered Psychometric Validity Screen [0.0]
  We tested whether seven instruction-tuned open-weight models produce verbalised confidence that meets minimal validity criteria.<n>We found Categorical elicitation did not rescue validity.<n> Token-level logprobability did not usefully predict verbalised confidence under the observed variance regime.
  arXiv  Detail & Related papers  (2026-04-24T04:45:21Z)
• MIRROR: A Hierarchical Benchmark for Metacognitive Calibration in Large Language Models [0.0]
  MIRROR is a benchmark that evaluates whether large language models can use self-knowledge to make better decisions.<n>We evaluate 16 models from 8 labs across approximately 250,000 evaluation instances.
  arXiv  Detail & Related papers  (2026-04-15T08:41:12Z)
• Correct Answers from Sound Reasoning: Verifiable Process Supervision for Language Models [94.68358825189738]
  We propose a post-training framework for verifiable domains that jointly optimize prediction accuracy and reasoning quality.<n>We evaluate VPS on chess, a controlled testbed where reasoning steps can be deterministically verified against engine signals.<n>VPS preserves accuracy while significantly improving reasoning quality, reducing win-rate error by up to 30% and restoring consistency to near saturation.
  arXiv  Detail & Related papers  (2026-04-03T15:19:46Z)
• Closing the Confidence-Faithfulness Gap in Large Language Models [3.122242683664973]
  Large language models verbalize confidence scores that are largely detached from their actual accuracy.<n>We present a mechanistic interpretability analysis of verbalized confidence.<n>We introduce a two-stage adaptive steering pipeline that reads the model's internal accuracy estimate and steers verbalized output to match it.
  arXiv  Detail & Related papers  (2026-03-26T05:42:04Z)
• On Calibration of Large Language Models: From Response To Capability [66.59139960234326]
  Large language models (LLMs) are widely deployed as general-purpose problem solvers.<n>We introduce capability calibration, which targets the model's expected accuracy on a query.<n>Our results demonstrate that capability-calibrated confidence improves pass@$k$ prediction and inference budget allocation.
  arXiv  Detail & Related papers  (2026-02-14T01:07:45Z)
• Modeling and Predicting Multi-Turn Answer Instability in Large Language Models [5.136381688235197]
  Large language models (LLMs) are adopted in an increasingly wide range of applications.<n>This paper employs simple multi-turn follow-up prompts to evaluate models' answer changes.<n>We find that a simple "Think again" prompt led to an approximate 10% accuracy drop for Gemini 1.5 Flash over nine turns.
  arXiv  Detail & Related papers  (2025-11-12T01:48:23Z)
• Robust Fine-tuning of Zero-shot Models via Variance Reduction [56.360865951192324]
  When fine-tuning zero-shot models, our desideratum is for the fine-tuned model to excel in both in-distribution (ID) and out-of-distribution (OOD) We propose a sample-wise ensembling technique that can simultaneously attain the best ID and OOD accuracy without the trade-offs.
  arXiv  Detail & Related papers  (2024-11-11T13:13:39Z)
• Towards Calibrated Robust Fine-Tuning of Vision-Language Models [97.19901765814431]
  This work proposes a robust fine-tuning method that improves both OOD accuracy and confidence calibration simultaneously in vision language models. We show that both OOD classification and OOD calibration errors have a shared upper bound consisting of two terms of ID data. Based on this insight, we design a novel framework that conducts fine-tuning with a constrained multimodal contrastive loss enforcing a larger smallest singular value.
  arXiv  Detail & Related papers  (2023-11-03T05:41:25Z)
• Conservative Prediction via Data-Driven Confidence Minimization [70.93946578046003]
  In safety-critical applications of machine learning, it is often desirable for a model to be conservative. We propose the Data-Driven Confidence Minimization framework, which minimizes confidence on an uncertainty dataset.
  arXiv  Detail & Related papers  (2023-06-08T07:05:36Z)
• Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction [49.15931834209624]
  We present the first benchmark that simulates the evaluation of open information extraction models in the real world.<n>We design and annotate a large-scale testbed in which each example is a knowledge-invariant clique.<n>By further elaborating the robustness metric, a model is judged to be robust if its performance is consistently accurate on the overall cliques.
  arXiv  Detail & Related papers  (2023-05-23T12:05:09Z)
• Meta-Learned Confidence for Few-shot Learning [60.6086305523402]
  A popular transductive inference technique for few-shot metric-based approaches, is to update the prototype of each class with the mean of the most confident query examples. We propose to meta-learn the confidence for each query sample, to assign optimal weights to unlabeled queries. We validate our few-shot learning model with meta-learned confidence on four benchmark datasets.
  arXiv  Detail & Related papers  (2020-02-27T10:22:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.