Dual Focal Loss for Calibration

• URL: http://arxiv.org/abs/2305.13665v1
• Date: Tue, 23 May 2023 04:19:16 GMT
• Title: Dual Focal Loss for Calibration
• Authors: Linwei Tao, Minjing Dong and Chang Xu
• Abstract summary: We propose a new loss function by focusing on dual logits. By maximizing the gap between these two logits, our proposed dual focal loss can achieve a better balance between over-confidence and under-confidence.
• Score: 21.663687352629225
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The use of deep neural networks in real-world applications require well-calibrated networks with confidence scores that accurately reflect the actual probability. However, it has been found that these networks often provide over-confident predictions, which leads to poor calibration. Recent efforts have sought to address this issue by focal loss to reduce over-confidence, but this approach can also lead to under-confident predictions. While different variants of focal loss have been explored, it is difficult to find a balance between over-confidence and under-confidence. In our work, we propose a new loss function by focusing on dual logits. Our method not only considers the ground truth logit, but also take into account the highest logit ranked after the ground truth logit. By maximizing the gap between these two logits, our proposed dual focal loss can achieve a better balance between over-confidence and under-confidence. We provide theoretical evidence to support our approach and demonstrate its effectiveness through evaluations on multiple models and datasets, where it achieves state-of-the-art performance. Code is available at https://github.com/Linwei94/DualFocalLoss

Related papers

• Revisiting Confidence Estimation: Towards Reliable Failure Prediction [53.79160907725975]
  We find a general, widely existing but actually-neglected phenomenon that most confidence estimation methods are harmful for detecting misclassification errors. We propose to enlarge the confidence gap by finding flat minima, which yields state-of-the-art failure prediction performance.
  arXiv  Detail & Related papers  (2024-03-05T11:44:14Z)
• Binary Classification with Confidence Difference [100.08818204756093]
  This paper delves into a novel weakly supervised binary classification problem called confidence-difference (ConfDiff) classification. We propose a risk-consistent approach to tackle this problem and show that the estimation error bound the optimal convergence rate. We also introduce a risk correction approach to mitigate overfitting problems, whose consistency and convergence rate are also proven.
  arXiv  Detail & Related papers  (2023-10-09T11:44:50Z)
• Rethinking Confidence Calibration for Failure Prediction [37.43981354073841]
  Modern deep neural networks are often overconfident for their incorrect predictions. We find that most confidence calibration methods are useless or harmful for failure prediction. We propose a simple hypothesis: flat minima is beneficial for failure prediction.
  arXiv  Detail & Related papers  (2023-03-06T08:54:18Z)
• Understanding Softmax Confidence and Uncertainty [95.71801498763216]
  It is often remarked that neural networks fail to increase their uncertainty when predicting on data far from the training distribution. Yet naively using softmax confidence as a proxy for uncertainty achieves modest success in tasks exclusively testing for this. This paper investigates this contradiction, identifying two implicit biases that do encourage softmax confidence to correlate with uncertainty.
  arXiv  Detail & Related papers  (2021-06-09T10:37:29Z)
• Learning to Cascade: Confidence Calibration for Improving the Accuracy and Computational Cost of Cascade Inference Systems [2.28438857884398]
  Deep neural networks are highly accurate but known to be overconfident. It is not clear whether confidence scores can improve the performance of systems that use confidence scores. We propose a new confidence calibration method, Learning to Cascade.
  arXiv  Detail & Related papers  (2021-04-15T07:09:09Z)
• Uncertainty-Aware Deep Calibrated Salient Object Detection [74.58153220370527]
  Existing deep neural network based salient object detection (SOD) methods mainly focus on pursuing high network accuracy. These methods overlook the gap between network accuracy and prediction confidence, known as the confidence uncalibration problem. We introduce an uncertaintyaware deep SOD network, and propose two strategies to prevent deep SOD networks from being overconfident.
  arXiv  Detail & Related papers  (2020-12-10T23:28:36Z)
• Confidence-aware Adversarial Learning for Self-supervised Semantic Matching [29.132600499226406]
  We introduce a Confidence-Aware Semantic Matching Network (CAMNet) First, we estimate a dense confidence map for a matching prediction through self-supervised learning. Second, based on the estimated confidence, we refine initial predictions by propagating reliable matching to the rest of locations on the image plane. We are the first that exploit confidence during refinement to improve semantic matching accuracy and develop an end-to-end self-supervised adversarial learning procedure for the entire matching network.
  arXiv  Detail & Related papers  (2020-08-25T09:15:48Z)
• Confidence-Aware Learning for Deep Neural Networks [4.9812879456945]
  We propose a method of training deep neural networks with a novel loss function, named Correctness Ranking Loss. It regularizes class probabilities explicitly to be better confidence estimates in terms of ordinal ranking according to confidence. It has almost the same computational costs for training as conventional deep classifiers and outputs reliable predictions by a single inference.
  arXiv  Detail & Related papers  (2020-07-03T02:00:35Z)
• Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks [65.24701908364383]
  We show that a sufficient condition for a uncertainty on a ReLU network is "to be a bit Bayesian calibrated" We further validate these findings empirically via various standard experiments using common deep ReLU networks and Laplace approximations.
  arXiv  Detail & Related papers  (2020-02-24T08:52:06Z)
• Calibrating Deep Neural Networks using Focal Loss [77.92765139898906]
  Miscalibration is a mismatch between a model's confidence and its correctness. We show that focal loss allows us to learn models that are already very well calibrated. We show that our approach achieves state-of-the-art calibration without compromising on accuracy in almost all cases.
  arXiv  Detail & Related papers  (2020-02-21T17:35:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.