FALCON-Discover: Discovering Concentrated False-Confidence Regions for Calibration

• URL: http://arxiv.org/abs/2607.18278v1
• Date: Mon, 29 Jun 2026 01:23:52 GMT
• Title: FALCON-Discover: Discovering Concentrated False-Confidence Regions for Calibration
• Abstract summary: We study false-confidence concentration, the extent to which confident errors occupy compact, discoverable regions of prediction space.<n>We introduce FALCON-Discover, a model-agnostic framework that ranks predictions using discrepancy signals from confidence, local support, neighborhood agreement, and stability.
• Score: 26.80740674805048
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Calibration is usually evaluated in aggregate, but the most dangerous failures are often local: predictions that remain highly confident despite being wrong. We study this failure mode as false-confidence concentration, the extent to which confident errors occupy compact, discoverable regions of prediction space. We introduce FALCON-Discover, a post-hoc, model-agnostic framework that ranks predictions using discrepancy signals from confidence, local support, neighborhood agreement, and perturbation stability. Across seven binary tabular datasets, four seeds, five-fold cross-fitting, and strong learners including XGBoost and CatBoost, we find that false-confidence concentration is recurrent but regime-dependent. At the main confidence threshold, discrepancy-based ranking substantially outperforms the strongest validation-selected calibration or trust-scoring baseline in the strongest regimes, while raw confidence recovers little dangerous-error mass. The best detector varies across datasets: learned discrepancy is strongest when multiple cues must be combined, whereas stability-centered ranking works best when local decisional fragility dominates. These results show that dangerous overconfidence is better treated as a family-level discovery problem than as a single-score calibration problem, and motivate calibration strategies that explicitly target regions where confidence, support, and stability diverge.

Related papers

• ReliableNet: A Chance-Constrained Approach to Trustworthy Classification in Deep Learning [0.0]
  A prediction that is both confident and wrong is a critical reliability failure.<n>We propose ReliableNet, which constrains the Joint Confident-Wrong (JCW) probability.<n> ReliableNet is the only method certified within the JCW budget for every dataset.
  arXiv  Detail & Related papers  (2026-08-10T15:58:19Z)
• Expectation Consistency Loss: Rethink Confidence Calibration under Covariate Shift [7.333172294727063]
  General confidence calibration methods assume training and test data are independent and identically distributed.<n>This paper proposes an unsupervised domain adaptation loss to calibrate confidence of the target domain.<n>We prove that computing ECL loss has the same sample complexity as Expected Error (ECE) and provide a theoretically grounded mini-batch trainable scheme for ECL loss.
  arXiv  Detail & Related papers  (2026-05-20T11:45:39Z)
• Discovery of Hidden Miscalibration Regimes [52.452902154360565]
  Models may be systematically overconfident on some kinds of inputs and underconfident on others.<n>We define the corresponding miscalibration field and propose a diagnostic framework for estimating it.<n>Our approach learns a calibration-aware representation of the input space and estimates signed local miscalibration by kernel smoothing in the learned geometry.
  arXiv  Detail & Related papers  (2026-05-13T13:07:50Z)
• Process Supervision of Confidence Margin for Calibrated LLM Reasoning [52.373121066425455]
  Scaling test-time computation with reinforcement learning (RL) has emerged as a reliable path to improve large language models (LLM) reasoning ability.<n>Yet, outcome-based reward often incentivizes models to be overconfident, leading to hallucinations, unreliable confidence-based control, and unnecessary compute allocation.<n>We introduce Reinforcement Learning with Confidence Margin (textbfRLCM), a calibration-aware RL framework that jointly optimize correctness and confidence reliability.
  arXiv  Detail & Related papers  (2026-04-25T14:40:13Z)
• Socrates Loss: Unifying Confidence Calibration and Classification by Leveraging the Unknown [4.503897051444244]
  Deep neural networks, despite their high accuracy, often exhibit poor confidence calibration, limiting their reliability in high-stakes applications.<n>Current ad-hoc confidence calibration methods attempt to fix this during training but face a fundamental trade-off.<n>This paper addresses and mitigates this stability-performance trade-off.<n>We propose Socrates Loss, a novel, unified loss function that explicitly leverages uncertainty by incorporating an auxiliary unknown class.
  arXiv  Detail & Related papers  (2026-04-14T03:43:15Z)
• A Confidence-Variance Theory for Pseudo-Label Selection in Semi-Supervised Learning [15.149171763610662]
  This paper introduces a Confidence-Variance (CoVar) theory framework that provides a principled joint reliability criterion for pseudo-label selection.<n>We show that reliable pseudo-labels should have both high MC and low RCV, and that the influence of RCV increases as confidence grows.<n>We integrate CoVar as a plug-in module into representative semi-supervised semantic segmentation and image classification methods.
  arXiv  Detail & Related papers  (2026-01-16T02:51:59Z)
• ConfTuner: Training Large Language Models to Express Their Confidence Verbally [58.63318088243125]
  Large Language Models (LLMs) are increasingly deployed in high-stakes domains such as science, law, and healthcare.<n>LLMs are often observed to generate incorrect answers with high confidence, a phenomenon known as "overconfidence"
  arXiv  Detail & Related papers  (2025-08-26T09:25:32Z)
• Enhance GNNs with Reliable Confidence Estimation via Adversarial Calibration Learning [30.450482094196243]
  Despite their impressive predictive performance, GNNs often exhibit poor confidence calibration.<n>This issue raises concerns about their reliability in high-stakes domains such as fraud detection, and risk assessment.<n>We propose a novel AdvCali framework that adaptively enhances calibration across different node groups.
  arXiv  Detail & Related papers  (2025-03-23T23:04:41Z)
• Fact-Level Confidence Calibration and Self-Correction [64.40105513819272]
  We propose a Fact-Level framework that calibrates confidence to relevance-weighted correctness at the fact level. We also develop Confidence-Guided Fact-level Self-Correction ($textbfConFix$), which uses high-confidence facts within a response as additional knowledge to improve low-confidence ones.
  arXiv  Detail & Related papers  (2024-11-20T14:15:18Z)
• Revisiting Confidence Estimation: Towards Reliable Failure Prediction [53.79160907725975]
  We find a general, widely existing but actually-neglected phenomenon that most confidence estimation methods are harmful for detecting misclassification errors. We propose to enlarge the confidence gap by finding flat minima, which yields state-of-the-art failure prediction performance.
  arXiv  Detail & Related papers  (2024-03-05T11:44:14Z)
• Binary Classification with Confidence Difference [100.08818204756093]
  This paper delves into a novel weakly supervised binary classification problem called confidence-difference (ConfDiff) classification. We propose a risk-consistent approach to tackle this problem and show that the estimation error bound the optimal convergence rate. We also introduce a risk correction approach to mitigate overfitting problems, whose consistency and convergence rate are also proven.
  arXiv  Detail & Related papers  (2023-10-09T11:44:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.