When Accuracy Is Not Enough: Uncertainty Collapse between Noisy Label Learning and Out-of-Distribution Detection

• URL: http://arxiv.org/abs/2605.17795v1
• Date: Mon, 18 May 2026 03:19:45 GMT
• Title: When Accuracy Is Not Enough: Uncertainty Collapse between Noisy Label Learning and Out-of-Distribution Detection
• Abstract summary: Learning with noisy labels (LNL) is typically benchmarked by closed-set classification accuracy.<n>We present a learner-agnostic ACC-OOD benchmark that freezes LNL checkpoints and evaluates them with standardized near-/far-OOD routing and post-hoc scores.<n>These results support LNL benchmarks that co-report closed-set generalization, open-world reliability, and structural overlap diagnostics.
• Score: 5.922967156812547
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Learning with noisy labels (LNL) is typically benchmarked by closed-set classification accuracy, yet deployment often requires classifiers to reject out-of-distribution (OOD) inputs. We present a learner-agnostic ACC-OOD benchmark that freezes LNL checkpoints and evaluates them with standardized near-/far-OOD routing and post-hoc scores across synthetic and real label noise. The benchmark reveals a recurring failure mode: high closed-set accuracy does not ensure OOD reliability, because low-confidence, misclassified in-distribution samples can overlap the score and feature regions occupied by OOD inputs under noisy training. We term this pathology uncertainty collapse. This structural overlap can make high-accuracy LNL methods lose separability at the ID-error/OOD interface under standard OOD scores. As an intervention, we study Virtual Margin Regularization (VMR), a lightweight repair probe demonstrated mainly with PSSCL that synthesizes boundary virtual outliers on trusted ID batches and widens the energy margin. VMR partially reduces the collapse-induced far-OOD failure without replacing the host objective or sacrificing closed-set accuracy in the tested settings. These results support LNL benchmarks that co-report closed-set generalization, open-world reliability, and structural overlap diagnostics.

Related papers

• Evaluating LLM Uncertainty in Long-Form Generation Using Deterministic Ground Truth [11.568922058618497]
  Long-form generation benchmarks tend to rely on fallible labels rather than deterministic ground truth.<n>We introduce Single-answer Atomic Long-form Target (SALT), a benchmark of six procedurally generated tasks with single deterministic long textual ground truths.<n>SALT enables controlled atom-level interventions throughout generation, revealing two separable drivers of future errors.
  arXiv  Detail & Related papers  (2026-07-04T13:25:18Z)
• Discovery of Hidden Miscalibration Regimes [52.452902154360565]
  Models may be systematically overconfident on some kinds of inputs and underconfident on others.<n>We define the corresponding miscalibration field and propose a diagnostic framework for estimating it.<n>Our approach learns a calibration-aware representation of the input space and estimates signed local miscalibration by kernel smoothing in the learned geometry.
  arXiv  Detail & Related papers  (2026-05-13T13:07:50Z)
• Beyond Surface Statistics: Robust Conformal Prediction for LLMs via Internal Representations [36.24681446361556]
  We propose a conformal framework for question answering that uses internal representations rather than output-facing statistics.<n>We show that our method achieves a better validity--efficiency trade-off than strong text-level baselines.<n>These results suggest that internal representations can provide more informative conformal scores when surface-level uncertainty is unstable under distribution shift.
  arXiv  Detail & Related papers  (2026-04-17T16:28:31Z)
• From Misclassifications to Outliers: Joint Reliability Assessment in Classification [13.95428115564986]
  A reliable system should not only detect out-of-distribution (OOD) inputs but also anticipate in-distribution (ID) errors.<n>We propose a unified evaluation framework that integrates OOD detection and failure prediction.<n>We introduce SURE+, a new approach that significantly improves reliability across diverse scenarios.
  arXiv  Detail & Related papers  (2026-03-04T10:11:51Z)
• USE: Uncertainty Structure Estimation for Robust Semi-Supervised Learning [6.021112703667433]
  Uncertainty Structure Estimation (USE) is introduced for Semi-supervised learning (SSL)<n>USE trains a proxy model on the labeled set to compute entropy scores for unlabeled samples.<n>It is evident that USE consistently improves accuracy and under varying levels of OOD contamination.
  arXiv  Detail & Related papers  (2026-02-28T01:31:46Z)
• Unsupervised Conformal Inference: Bootstrapping and Alignment to Control LLM Uncertainty [49.19257648205146]
  We propose an unsupervised conformal inference framework for generation.<n>Our gates achieve close-to-nominal coverage and provide tighter, more stable thresholds than split UCP.<n>The result is a label-free, API-compatible gate for test-time filtering.
  arXiv  Detail & Related papers  (2025-09-26T23:40:47Z)
• TrustLoRA: Low-Rank Adaptation for Failure Detection under Out-of-distribution Data [62.22804234013273]
  We propose a simple failure detection framework to unify and facilitate classification with rejection under both covariate and semantic shifts.<n>Our key insight is that by separating and consolidating failure-specific reliability knowledge with low-rank adapters, we can enhance the failure detection ability effectively and flexibly.
  arXiv  Detail & Related papers  (2025-04-20T09:20:55Z)
• Are Domain Generalization Benchmarks with Accuracy on the Line Misspecified? [11.534630666670568]
  Spurious correlations, unstable statistical shortcuts a model can exploit, are expected to degrade performance out-of-distribution.<n>We show that current practice evaluates "robustness" without truly stressing the spurious signals we seek to eliminate.
  arXiv  Detail & Related papers  (2025-03-31T19:50:04Z)
• How Does Unlabeled Data Provably Help Out-of-Distribution Detection? [63.41681272937562]
  Unlabeled in-the-wild data is non-trivial due to the heterogeneity of both in-distribution (ID) and out-of-distribution (OOD) data. This paper introduces a new learning framework SAL (Separate And Learn) that offers both strong theoretical guarantees and empirical effectiveness.
  arXiv  Detail & Related papers  (2024-02-05T20:36:33Z)
• Threshold-Consistent Margin Loss for Open-World Deep Metric Learning [42.03620337000911]
  Existing losses used in deep metric learning (DML) for image retrieval often lead to highly non-uniform intra-class and inter-class representation structures. Inconsistency often complicates the threshold selection process when deploying commercial image retrieval systems. We propose a novel variance-based metric called Operating-Point-Inconsistency-Score (OPIS) that quantifies the variance in the operating characteristics across classes.
  arXiv  Detail & Related papers  (2023-07-08T21:16:41Z)
• Provably Robust Detection of Out-of-distribution Data (almost) for free [124.14121487542613]
  Deep neural networks are known to produce highly overconfident predictions on out-of-distribution (OOD) data. In this paper we propose a novel method where from first principles we combine a certifiable OOD detector with a standard classifier into an OOD aware classifier. In this way we achieve the best of two worlds: certifiably adversarially robust OOD detection, even for OOD samples close to the in-distribution, without loss in prediction accuracy and close to state-of-the-art OOD detection performance for non-manipulated OOD data.
  arXiv  Detail & Related papers  (2021-06-08T11:40:49Z)
• Certifiably Adversarially Robust Detection of Out-of-Distribution Data [111.67388500330273]
  We aim for certifiable worst case guarantees for OOD detection by enforcing low confidence at the OOD point. We show that non-trivial bounds on the confidence for OOD data generalizing beyond the OOD dataset seen at training time are possible.
  arXiv  Detail & Related papers  (2020-07-16T17:16:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.