Towards Accurate and Calibrated Classification: Regularizing Cross-Entropy From A Generative Perspective

• URL: http://arxiv.org/abs/2604.06689v1
• Date: Wed, 08 Apr 2026 05:11:00 GMT
• Title: Towards Accurate and Calibrated Classification: Regularizing Cross-Entropy From A Generative Perspective
• Abstract summary: Generative Cross-Entropy (GCE) is equivalent to cross-entropy augmented with a class-level confidence regularizer.<n>GCE attains calibration competitive with focal-loss variants without sacrificing accuracy.
• Score: 11.480485886662327
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate classification requires not only high predictive accuracy but also well-calibrated confidence estimates. Yet, modern deep neural networks (DNNs) are often overconfident, primarily due to overfitting on the negative log-likelihood (NLL). While focal loss variants alleviate this issue, they typically reduce accuracy, revealing a persistent trade-off between calibration and predictive performance. Motivated by the complementary strengths of generative and discriminative classifiers, we propose Generative Cross-Entropy (GCE), which maximizes $p(x|y)$ and is equivalent to cross-entropy augmented with a class-level confidence regularizer. Under mild conditions, GCE is strictly proper. Across CIFAR-10/100, Tiny-ImageNet, and a medical imaging benchmark, GCE improves both accuracy and calibration over cross-entropy, especially in the long-tailed scenario. Combined with adaptive piecewise temperature scaling (ATS), GCE attains calibration competitive with focal-loss variants without sacrificing accuracy.

Related papers

• Margin and Consistency Supervision for Calibrated and Robust Vision Models [1.0152838128195467]
  Margin and Consistency Supervision (MaCS) is a simple, architecture-agnostic regularization framework.<n>MaCS jointly enforces logit-space separation and local prediction stability.<n>We show that increasing classification margin while reducing local sensitivity formalized via a Lipschitz-type stability proxy yields improved generalization guarantees.
  arXiv  Detail & Related papers  (2026-03-06T01:54:28Z)
• LATA: Laplacian-Assisted Transductive Adaptation for Conformal Uncertainty in Medical VLMs [61.06744611795341]
  Medical vision-language models (VLMs) are strong zero-shot recognizers for medical imaging.<n>We propose texttttextbfLATA (Laplacian-Assisted Transductive Adaptation), a textittraining- and label-free refinement.<n>texttttextbfLATA sharpens zero-shot predictions without compromising exchangeability.
  arXiv  Detail & Related papers  (2026-02-19T16:45:38Z)
• Deep Neural Network Calibration by Reducing Classifier Shift with Stochastic Masking [24.347895497146876]
  Deep neural networks (DNNs) often suffer from poor calibration, especially in safety-critical scenarios such as autonomous driving and healthcare.<n>We propose MaC-Cal, a novel mask-based calibration method that leverages sparsity to enhance the alignment between confidence and accuracy.<n>MaC-Cal achieves superior calibration performance and robustness under data corruption.
  arXiv  Detail & Related papers  (2025-08-12T17:50:23Z)
• The Final Layer Holds the Key: A Unified and Efficient GNN Calibration Framework [31.107286245419044]
  Graph Neural Networks (GNNs) have demonstrated remarkable effectiveness on graph-based tasks.<n>However, their predictive confidence is often miscalibrated, typically exhibiting under-confidence.<n>We propose a simple yet efficient graph calibration method to address this issue.
  arXiv  Detail & Related papers  (2025-05-16T15:02:17Z)
• Towards Calibrated Deep Clustering Network [60.71776081164377]
  In deep clustering, the estimated confidence for a sample belonging to a particular cluster greatly exceeds its actual prediction accuracy.<n>We propose a novel dual head (calibration head and clustering head) deep clustering model that can effectively calibrate the estimated confidence and the actual accuracy.<n>The proposed calibrated deep clustering model not only surpasses the state-of-the-art deep clustering methods by 5x on average in terms of expected calibration error, but also significantly outperforms them in terms of clustering accuracy.
  arXiv  Detail & Related papers  (2024-03-04T11:23:40Z)
• How to Fix a Broken Confidence Estimator: Evaluating Post-hoc Methods for Selective Classification with Deep Neural Networks [1.4502611532302039]
  We show that a simple $p$-norm normalization of the logits, followed by taking the maximum logit as the confidence estimator, can lead to considerable gains in selective classification performance. Our results are shown to be consistent under distribution shift.
  arXiv  Detail & Related papers  (2023-05-24T18:56:55Z)
• Bridging Precision and Confidence: A Train-Time Loss for Calibrating Object Detection [58.789823426981044]
  We propose a novel auxiliary loss formulation that aims to align the class confidence of bounding boxes with the accurateness of predictions. Our results reveal that our train-time loss surpasses strong calibration baselines in reducing calibration error for both in and out-domain scenarios.
  arXiv  Detail & Related papers  (2023-03-25T08:56:21Z)
• AdaFocal: Calibration-aware Adaptive Focal Loss [8.998525155518836]
  Training with focal loss leads to better calibration than cross-entropy. We propose a calibration-aware adaptive focal loss called AdaFocal.
  arXiv  Detail & Related papers  (2022-11-21T20:19:24Z)
• Beyond calibration: estimating the grouping loss of modern neural networks [68.8204255655161]
  Proper scoring rule theory shows that given the calibration loss, the missing piece to characterize individual errors is the grouping loss. We show that modern neural network architectures in vision and NLP exhibit grouping loss, notably in distribution shifts settings.
  arXiv  Detail & Related papers  (2022-10-28T07:04:20Z)
• Sample-dependent Adaptive Temperature Scaling for Improved Calibration [95.7477042886242]
  Post-hoc approach to compensate for neural networks being wrong is to perform temperature scaling. We propose to predict a different temperature value for each input, allowing us to adjust the mismatch between confidence and accuracy. We test our method on the ResNet50 and WideResNet28-10 architectures using the CIFAR10/100 and Tiny-ImageNet datasets.
  arXiv  Detail & Related papers  (2022-07-13T14:13:49Z)
• Calibrating Deep Neural Networks using Focal Loss [77.92765139898906]
  Miscalibration is a mismatch between a model's confidence and its correctness. We show that focal loss allows us to learn models that are already very well calibrated. We show that our approach achieves state-of-the-art calibration without compromising on accuracy in almost all cases.
  arXiv  Detail & Related papers  (2020-02-21T17:35:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.