Neural Clamping: Joint Input Perturbation and Temperature Scaling for Neural Network Calibration

• URL: http://arxiv.org/abs/2209.11604v2
• Date: Wed, 24 Jul 2024 20:47:55 GMT
• Title: Neural Clamping: Joint Input Perturbation and Temperature Scaling for Neural Network Calibration
• Authors: Yung-Chen Tang, Pin-Yu Chen, Tsung-Yi Ho,
• Abstract summary: We propose a new post-processing calibration method called Neural Clamping. Our empirical results show that Neural Clamping significantly outperforms state-of-the-art post-processing calibration methods.
• Score: 62.4971588282174
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural network calibration is an essential task in deep learning to ensure consistency between the confidence of model prediction and the true correctness likelihood. In this paper, we propose a new post-processing calibration method called Neural Clamping, which employs a simple joint input-output transformation on a pre-trained classifier via a learnable universal input perturbation and an output temperature scaling parameter. Moreover, we provide theoretical explanations on why Neural Clamping is provably better than temperature scaling. Evaluated on BloodMNIST, CIFAR-100, and ImageNet image recognition datasets and a variety of deep neural network models, our empirical results show that Neural Clamping significantly outperforms state-of-the-art post-processing calibration methods. The code is available at github.com/yungchentang/NCToolkit, and the demo is available at huggingface.co/spaces/TrustSafeAI/NCTV.

Related papers

• Rolling bearing fault diagnosis method based on generative adversarial enhanced multi-scale convolutional neural network model [7.600902237804825]
  A rolling bearing fault diagnosis method based on generative adversarial enhanced multi-scale convolutional neural network model is proposed. Compared with ResNet method, the experimental results show that the proposed method has better generalization performance and anti-noise performance.
  arXiv  Detail & Related papers  (2024-03-21T06:42:35Z)
• Neural Priming for Sample-Efficient Adaptation [92.14357804106787]
  We propose Neural Priming, a technique for adapting large pretrained models to distribution shifts and downstream tasks. Neural Priming can be performed at test time, even for pretraining as large as LAION-2B.
  arXiv  Detail & Related papers  (2023-06-16T21:53:16Z)
• A Deep Learning-based in silico Framework for Optimization on Retinal Prosthetic Stimulation [3.870538485112487]
  We propose a neural network-based framework to optimize the perceptions simulated by the in silico retinal implant model pulse2percept. The pipeline consists of a trainable encoder, a pre-trained retinal implant model and a pre-trained evaluator.
  arXiv  Detail & Related papers  (2023-02-07T16:32:05Z)
• NCTV: Neural Clamping Toolkit and Visualization for Neural Network Calibration [66.22668336495175]
  A lack of consideration for neural network calibration will not gain trust from humans. We introduce the Neural Clamping Toolkit, the first open-source framework designed to help developers employ state-of-the-art model-agnostic calibrated models.
  arXiv  Detail & Related papers  (2022-11-29T15:03:05Z)
• Sample-dependent Adaptive Temperature Scaling for Improved Calibration [95.7477042886242]
  Post-hoc approach to compensate for neural networks being wrong is to perform temperature scaling. We propose to predict a different temperature value for each input, allowing us to adjust the mismatch between confidence and accuracy. We test our method on the ResNet50 and WideResNet28-10 architectures using the CIFAR10/100 and Tiny-ImageNet datasets.
  arXiv  Detail & Related papers  (2022-07-13T14:13:49Z)
• Neuron-based Pruning of Deep Neural Networks with Better Generalization using Kronecker Factored Curvature Approximation [18.224344440110862]
  The proposed algorithm directs the parameters of the compressed model toward a flatter solution by exploring the spectral radius of Hessian. Our result shows that it improves the state-of-the-art results on neuron compression. The method is able to achieve very small networks with small accuracy across different neural network models.
  arXiv  Detail & Related papers  (2021-11-16T15:55:59Z)
• Intra Order-preserving Functions for Calibration of Multi-Class Neural Networks [54.23874144090228]
  A common approach is to learn a post-hoc calibration function that transforms the output of the original network into calibrated confidence scores. Previous post-hoc calibration techniques work only with simple calibration functions. We propose a new neural network architecture that represents a class of intra order-preserving functions.
  arXiv  Detail & Related papers  (2020-03-15T12:57:21Z)
• Calibrating Deep Neural Networks using Focal Loss [77.92765139898906]
  Miscalibration is a mismatch between a model's confidence and its correctness. We show that focal loss allows us to learn models that are already very well calibrated. We show that our approach achieves state-of-the-art calibration without compromising on accuracy in almost all cases.
  arXiv  Detail & Related papers  (2020-02-21T17:35:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.