NCTV: Neural Clamping Toolkit and Visualization for Neural Network Calibration

• URL: http://arxiv.org/abs/2211.16274v1
• Date: Tue, 29 Nov 2022 15:03:05 GMT
• Title: NCTV: Neural Clamping Toolkit and Visualization for Neural Network Calibration
• Authors: Lei Hsiung, Yung-Chen Tang, Pin-Yu Chen, Tsung-Yi Ho
• Abstract summary: A lack of consideration for neural network calibration will not gain trust from humans. We introduce the Neural Clamping Toolkit, the first open-source framework designed to help developers employ state-of-the-art model-agnostic calibrated models.
• Score: 66.22668336495175
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the advancement of deep learning technology, neural networks have demonstrated their excellent ability to provide accurate predictions in many tasks. However, a lack of consideration for neural network calibration will not gain trust from humans, even for high-accuracy models. In this regard, the gap between the confidence of the model's predictions and the actual correctness likelihood must be bridged to derive a well-calibrated model. In this paper, we introduce the Neural Clamping Toolkit, the first open-source framework designed to help developers employ state-of-the-art model-agnostic calibrated models. Furthermore, we provide animations and interactive sections in the demonstration to familiarize researchers with calibration in neural networks. A Colab tutorial on utilizing our toolkit is also introduced.

Related papers

• What Can the Neural Tangent Kernel Tell Us About Adversarial Robustness? [0.0]
  We study adversarial examples of trained neural networks through analytical tools afforded by recent theory advances connecting neural networks and kernel methods. We show how NTKs allow to generate adversarial examples in a training-free'' fashion, and demonstrate that they transfer to fool their finite-width neural net counterparts in the lazy'' regime.
  arXiv  Detail & Related papers  (2022-10-11T16:11:48Z)
• Bayesian Neural Network Versus Ex-Post Calibration For Prediction Uncertainty [0.2343856409260935]
  Probabilistic predictions from neural networks account for predictive uncertainty during classification. In practice most datasets are trained on non-probabilistic neural networks which by default do not capture this inherent uncertainty. A plausible alternative to the calibration approach is to use Bayesian neural networks, which directly models a predictive distribution.
  arXiv  Detail & Related papers  (2022-09-29T07:22:19Z)
• Neural Clamping: Joint Input Perturbation and Temperature Scaling for Neural Network Calibration [62.4971588282174]
  We propose a new post-processing calibration method called Neural Clamping. Our empirical results show that Neural Clamping significantly outperforms state-of-the-art post-processing calibration methods.
  arXiv  Detail & Related papers  (2022-09-23T14:18:39Z)
• Can pruning improve certified robustness of neural networks? [106.03070538582222]
  We show that neural network pruning can improve empirical robustness of deep neural networks (NNs) Our experiments show that by appropriately pruning an NN, its certified accuracy can be boosted up to 8.2% under standard training. We additionally observe the existence of certified lottery tickets that can match both standard and certified robust accuracies of the original dense models.
  arXiv  Detail & Related papers  (2022-06-15T05:48:51Z)
• On Calibration of Graph Neural Networks for Node Classification [29.738179864433445]
  Graph neural networks learn entity and edge embeddings for tasks such as node classification and link prediction. These models achieve good performance with respect to accuracy, but the confidence scores associated with the predictions might not be calibrated. We propose a topology-aware calibration method that takes the neighboring nodes into account and yields improved calibration.
  arXiv  Detail & Related papers  (2022-06-03T13:48:10Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• The Neural Coding Framework for Learning Generative Models [91.0357317238509]
  We propose a novel neural generative model inspired by the theory of predictive processing in the brain. In a similar way, artificial neurons in our generative model predict what neighboring neurons will do, and adjust their parameters based on how well the predictions matched reality.
  arXiv  Detail & Related papers  (2020-12-07T01:20:38Z)
• A Bayesian Perspective on Training Speed and Model Selection [51.15664724311443]
  We show that a measure of a model's training speed can be used to estimate its marginal likelihood. We verify our results in model selection tasks for linear models and for the infinite-width limit of deep neural networks. Our results suggest a promising new direction towards explaining why neural networks trained with gradient descent are biased towards functions that generalize well.
  arXiv  Detail & Related papers  (2020-10-27T17:56:14Z)
• Consistent feature selection for neural networks via Adaptive Group Lasso [3.42658286826597]
  We propose and establish a theoretical guarantee for the use of the adaptive group for selecting important features of neural networks. Specifically, we show that our feature selection method is consistent for single-output feed-forward neural networks with one hidden layer and hyperbolic tangent activation function.
  arXiv  Detail & Related papers  (2020-05-30T18:50:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.