Hidden Failures in Robustness: Why Supervised Uncertainty Quantification Needs Better Evaluation

• URL: http://arxiv.org/abs/2604.11662v1
• Date: Mon, 13 Apr 2026 16:11:33 GMT
• Title: Hidden Failures in Robustness: Why Supervised Uncertainty Quantification Needs Better Evaluation
• Abstract summary: Recent work has shown that the hidden states of large language models contain signals useful for uncertainty estimation and hallucination detection.<n>Yet it remains unclear how robust existing methods are, and which probe designs provide uncertainty estimates that are reliable under distribution shift.<n>We present a systematic study of supervised uncertainty probes across models, tasks, and OOD settings, training over 2,000 probes while varying the representation layer, feature type, and token aggregation strategy.
• Score: 40.069986955093526
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent work has shown that the hidden states of large language models contain signals useful for uncertainty estimation and hallucination detection, motivating a growing interest in efficient probe-based approaches. Yet it remains unclear how robust existing methods are, and which probe designs provide uncertainty estimates that are reliable under distribution shift. We present a systematic study of supervised uncertainty probes across models, tasks, and OOD settings, training over 2,000 probes while varying the representation layer, feature type, and token aggregation strategy. Our evaluation highlights poor robustness in current methods, particularly in the case of long-form generations. We also find that probe robustness is driven less by architecture and more by the probe inputs. Middle-layer representations generalise more reliably than final-layer hidden states, and aggregating across response tokens is consistently more robust than relying on single-token features. These differences are often largely invisible in-distribution but become more important under distribution shift. Informed by our evaluation, we explore a simple hybrid back-off strategy for improving robustness, arguing that better evaluation is a prerequisite for building more robust probes.

Related papers

• From Signals to Transfer: A Factorised Study of Probe-Based Uncertainty Estimation in Large Language Models [51.470637128942336]
  We propose a factorised study of probe-based uncertainty estimation (UE) under matched conditions.<n>Our results show that raw hidden states and attention features are difficult to outperform in-domain.<n>We train benchmark-based pretrained probes that transfer reasonably well to open-ended factual generation.
  arXiv  Detail & Related papers  (2026-06-26T03:23:15Z)
• From Passive Metric to Active Signal: The Evolving Role of Uncertainty Quantification in Large Language Models [77.04403907729738]
  This survey charts the evolution of uncertainty from a passive diagnostic metric to an active control signal guiding real-time model behavior.<n>We demonstrate how uncertainty is leveraged as an active control signal across three frontiers.<n>This survey argues that mastering the new trend of uncertainty is essential for building the next generation of scalable, reliable, and trustworthy AI.
  arXiv  Detail & Related papers  (2026-01-22T06:21:31Z)
• Rethinking Cross-Generator Image Forgery Detection through DINOv3 [62.80415066351157]
  Cross-generator detection has emerged as a new challenge forgenerative models.<n>We show that frozen visual foundation models, especially DINOv3, already exhibit strong cross-generator detection capability.<n>We introduce a training-free token-ranking strategy followed by a lightweight linear probe to select a small subset of authenticity-relevant tokens.
  arXiv  Detail & Related papers  (2025-11-27T14:01:50Z)
• Deep Learning Models for Robust Facial Liveness Detection [56.08694048252482]
  This study introduces a robust solution through novel deep learning models addressing the deficiencies in contemporary anti-spoofing techniques.<n>By innovatively integrating texture analysis and reflective properties associated with genuine human traits, our models distinguish authentic presence from replicas with remarkable precision.
  arXiv  Detail & Related papers  (2025-08-12T17:19:20Z)
• Interpretable Failure Detection with Human-Level Concepts [9.086637005449672]
  This research introduces an innovative strategy, leveraging human-level concepts for a dual purpose: to reliably detect when a model fails and to transparently interpret why.<n>We present a simple yet highly effective approach based on the ordinal ranking of concept activation to the input image.
  arXiv  Detail & Related papers  (2025-02-07T19:19:40Z)
• Reliability in Semantic Segmentation: Are We on the Right Track? [15.0189654919665]
  We analyze a broad variety of models, spanning from older ResNet-based architectures to novel transformers. We find that while recent models are significantly more robust, they are not overall more reliable in terms of uncertainty estimation. This is the first study on modern segmentation models focused on both robustness and uncertainty estimation.
  arXiv  Detail & Related papers  (2023-03-20T17:38:24Z)
• How Reliable is Your Regression Model's Uncertainty Under Real-World Distribution Shifts? [46.05502630457458]
  We propose a benchmark of 8 image-based regression datasets with different types of challenging distribution shifts. We find that while methods are well calibrated when there is no distribution shift, they all become highly overconfident on many of the benchmark datasets.
  arXiv  Detail & Related papers  (2023-02-07T18:54:39Z)
• Residual Error: a New Performance Measure for Adversarial Robustness [85.0371352689919]
  A major challenge that limits the wide-spread adoption of deep learning has been their fragility to adversarial attacks. This study presents the concept of residual error, a new performance measure for assessing the adversarial robustness of a deep neural network. Experimental results using the case of image classification demonstrate the effectiveness and efficacy of the proposed residual error metric.
  arXiv  Detail & Related papers  (2021-06-18T16:34:23Z)
• Adversarial Robustness under Long-Tailed Distribution [93.50792075460336]
  Adversarial robustness has attracted extensive studies recently by revealing the vulnerability and intrinsic characteristics of deep networks. In this work we investigate the adversarial vulnerability as well as defense under long-tailed distributions. We propose a clean yet effective framework, RoBal, which consists of two dedicated modules, a scale-invariant and data re-balancing.
  arXiv  Detail & Related papers  (2021-04-06T17:53:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.