Deep Feature Response Discriminative Calibration

• URL: http://arxiv.org/abs/2411.13582v1
• Date: Sat, 16 Nov 2024 10:48:32 GMT
• Title: Deep Feature Response Discriminative Calibration
• Authors: Wenxiang Xu, Tian Qiu, Linyun Zhou, Zunlei Feng, Mingli Song, Huiqiong Wang,
• Abstract summary: optimization techniques, such as ResNet and SENet, have been proposed to improve model accuracy. They lack the discriminative calibration for different features, thereby introducing limitations in the model output. We propose a method that discriminatively calibrates feature responses.
• Score: 37.81540706307031
• License:
• Abstract: Deep neural networks (DNNs) have numerous applications across various domains. Several optimization techniques, such as ResNet and SENet, have been proposed to improve model accuracy. These techniques improve the model performance by adjusting or calibrating feature responses according to a uniform standard. However, they lack the discriminative calibration for different features, thereby introducing limitations in the model output. Therefore, we propose a method that discriminatively calibrates feature responses. The preliminary experimental results indicate that the neural feature response follows a Gaussian distribution. Consequently, we compute confidence values by employing the Gaussian probability density function, and then integrate these values with the original response values. The objective of this integration is to improve the feature discriminability of the neural feature response. Based on the calibration values, we propose a plugin-based calibration module incorporated into a modified ResNet architecture, termed Response Calibration Networks (ResCNet). Extensive experiments on datasets like CIFAR-10, CIFAR-100, SVHN, and ImageNet demonstrate the effectiveness of the proposed approach. The developed code is publicly available at https://github.com/tcmyxc/ResCNet.

Related papers

• Feature Clipping for Uncertainty Calibration [24.465567005078135]
  Modern deep neural networks (DNNs) often suffer from overconfidence, leading to miscalibration. We propose a novel post-hoc calibration method called feature clipping (FC) to address this issue. FC involves clipping feature values to a specified threshold, effectively increasing entropy in high calibration error samples.
  arXiv  Detail & Related papers  (2024-10-16T06:44:35Z)
• Probabilistic MIMO U-Net: Efficient and Accurate Uncertainty Estimation for Pixel-wise Regression [1.4528189330418977]
  Uncertainty estimation in machine learning is paramount for enhancing the reliability and interpretability of predictive models. We present an adaptation of the Multiple-Input Multiple-Output (MIMO) framework for pixel-wise regression tasks.
  arXiv  Detail & Related papers  (2023-08-14T22:08:28Z)
• Bridging Precision and Confidence: A Train-Time Loss for Calibrating Object Detection [58.789823426981044]
  We propose a novel auxiliary loss formulation that aims to align the class confidence of bounding boxes with the accurateness of predictions. Our results reveal that our train-time loss surpasses strong calibration baselines in reducing calibration error for both in and out-domain scenarios.
  arXiv  Detail & Related papers  (2023-03-25T08:56:21Z)
• Neural Clamping: Joint Input Perturbation and Temperature Scaling for Neural Network Calibration [62.4971588282174]
  We propose a new post-processing calibration method called Neural Clamping. Our empirical results show that Neural Clamping significantly outperforms state-of-the-art post-processing calibration methods.
  arXiv  Detail & Related papers  (2022-09-23T14:18:39Z)
• Meta-Solver for Neural Ordinary Differential Equations [77.8918415523446]
  We investigate how the variability in solvers' space can improve neural ODEs performance. We show that the right choice of solver parameterization can significantly affect neural ODEs models in terms of robustness to adversarial attacks.
  arXiv  Detail & Related papers  (2021-03-15T17:26:34Z)
• Insta-RS: Instance-wise Randomized Smoothing for Improved Robustness and Accuracy [9.50143683501477]
  Insta-RS is a multiple-start search algorithm that assigns customized Gaussian variances to test examples. Insta-RS Train is a novel two-stage training algorithm that adaptively adjusts and customizes the noise level of each training example. We show that our method significantly enhances the average certified radius (ACR) as well as the clean data accuracy.
  arXiv  Detail & Related papers  (2021-03-07T19:46:07Z)
• Neural Model-based Optimization with Right-Censored Observations [42.530925002607376]
  Neural networks (NNs) have been demonstrated to work well at the core of model-based optimization procedures. We show that our trained regression models achieve a better predictive quality than several baselines.
  arXiv  Detail & Related papers  (2020-09-29T07:32:30Z)
• On Calibration of Mixup Training for Deep Neural Networks [1.6242924916178283]
  We argue and provide empirical evidence that, due to its fundamentals, Mixup does not necessarily improve calibration. Our loss is inspired by Bayes decision theory and introduces a new training framework for designing losses for probabilistic modelling. We provide state-of-the-art accuracy with consistent improvements in calibration performance.
  arXiv  Detail & Related papers  (2020-03-22T16:54:31Z)
• Intra Order-preserving Functions for Calibration of Multi-Class Neural Networks [54.23874144090228]
  A common approach is to learn a post-hoc calibration function that transforms the output of the original network into calibrated confidence scores. Previous post-hoc calibration techniques work only with simple calibration functions. We propose a new neural network architecture that represents a class of intra order-preserving functions.
  arXiv  Detail & Related papers  (2020-03-15T12:57:21Z)
• Calibrating Deep Neural Networks using Focal Loss [77.92765139898906]
  Miscalibration is a mismatch between a model's confidence and its correctness. We show that focal loss allows us to learn models that are already very well calibrated. We show that our approach achieves state-of-the-art calibration without compromising on accuracy in almost all cases.
  arXiv  Detail & Related papers  (2020-02-21T17:35:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.