LiST: Lipschitz Scaling Training for Robust and Calibrated Neural Networks

• URL: http://arxiv.org/abs/2607.07745v1
• Date: Wed, 08 Jul 2026 08:34:01 GMT
• Title: LiST: Lipschitz Scaling Training for Robust and Calibrated Neural Networks
• Abstract summary: Lipschitz-constrained models guarantee robustness by design, yet the manual selection of the Lipschitz constraint L governs the resulting accuracy-robustness trade-off.<n>We show a theoretical and empirical link between the enforced Lipschitz constraint and Temperature Scaling, a state-of-the-art calibration method.<n>We introduce Lipschitz Scaling Training (LiST), a novel training paradigm that iteratively adjusts the global Lipschitz constant to reach this operating point.
• Score: 5.0813117839734145
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While accuracy, robustness, and calibration are all essential for reliable neural networks, they are often studied separately; developing models that satisfy all three simultaneously remains a central challenge. Lipschitz-constrained models guarantee robustness by design, yet the manual selection of the Lipschitz constraint L governs the resulting accuracy-robustness trade-off, and their calibration properties remain largely underexplored. In this work, we highlight a theoretical and empirical link between the enforced Lipschitz constraint and Temperature Scaling, a state-of-the-art calibration method. Specifically, we find that for a given training scheme, there exists a non-trivial value L* that yields an out-of-the-box calibrated network, and that calibration acts as a principled criterion to select a well-defined operating point on the accuracy-robustness Pareto front. Leveraging these insights, we introduce Lipschitz Scaling Training (LiST), a novel training paradigm that iteratively adjusts the global Lipschitz constant to reach this operating point. Through a margin parameter in the training loss, LiST further enables the construction of a fully calibrated Pareto front, allowing users to navigate the accuracy-robustness trade-off while remaining calibrated throughout. At convergence, LiST also enables the reintegration of calibration data into training, improving sample efficiency without sacrificing calibration. We validate LiST on CIFAR-10/100 and Tiny-ImageNet, demonstrating competitive accuracy and robustness against constrained and unconstrained baselines, while remaining calibrated out of the box. Code is available at GitHub.

Related papers

• Bellman Calibration for Marginalized Importance Weighting in Offline Reinforcement Learning [40.322273308230606]
  We introduce isotonic Bellman calibration, a one-dimensional, model-agnostic post-processing method.<n>Method corrects estimate's scale and shape by applying fitted occupancy-ratio evaluation (FORE) over a one-dimensional class of nondecreasing transformations.<n>We derive a calibration-refinement bound showing that any fitted ratio with small calibration error performs nearly as well as the best post-processing.
  arXiv  Detail & Related papers  (2026-08-25T17:44:02Z)
• Too Sharp, Too Sure: When Calibration Follows Curvature [39.687359662779826]
  Empirically, Expected Error (ECE) closely tracks curvature-based sharpness throughout optimization.<n>We show that both ECE and Gauss-Newton curvature are controlled, up to problem-specific constants.<n>We introduce a margin-aware training objective that explicitly targets robust-margin tails and local smoothness.
  arXiv  Detail & Related papers  (2026-04-22T14:28:52Z)
• Uniform convergence of the smooth calibration error and its relationship with functional gradient [15.875913304310297]
  This work focuses on the smooth calibration error (CE) and provides a uniform convergence bound.<n>We analyze three representative algorithms: gradient boosting trees, kernel boosting, and two-layer neural networks.<n>Our results offer new theoretical insights and practical guidance for designing reliable probabilistic models.
  arXiv  Detail & Related papers  (2025-05-26T01:23:56Z)
• Unconstrained Monotonic Calibration of Predictions in Deep Ranking Systems [29.90543561470141]
  A ranking model's absolute values are essential for certain downstream tasks.<n>Existing calibration approaches typically employ predefined transformation functions with order-preserving properties to adjust the original predictions.<n>We propose implementing a calibrator using an Unconstrained Monotonic Neural Network (UMNN), which can learn arbitrary monotonic functions.<n>This approach significantly relaxes the constraints on the calibrator, improving its flexibility and expressiveness while avoiding excessively distorting the original predictions.
  arXiv  Detail & Related papers  (2025-04-19T09:35:11Z)
• Towards Certification of Uncertainty Calibration under Adversarial Attacks [96.48317453951418]
  We show that attacks can significantly harm calibration, and thus propose certified calibration as worst-case bounds on calibration under adversarial perturbations.<n>We propose novel calibration attacks and demonstrate how they can improve model calibration through textitadversarial calibration training
  arXiv  Detail & Related papers  (2024-05-22T18:52:09Z)
• Calibration by Distribution Matching: Trainable Kernel Calibration Metrics [56.629245030893685]
  We introduce kernel-based calibration metrics that unify and generalize popular forms of calibration for both classification and regression. These metrics admit differentiable sample estimates, making it easy to incorporate a calibration objective into empirical risk minimization. We provide intuitive mechanisms to tailor calibration metrics to a decision task, and enforce accurate loss estimation and no regret decisions.
  arXiv  Detail & Related papers  (2023-10-31T06:19:40Z)
• Calibrated and Sharp Uncertainties in Deep Learning via Density Estimation [10.209143402485406]
  This paper argues that calibration is important in practice and is easy to maintain.<n>We introduce a simple training procedure based on recalibration that yields calibrated models without sacrificing overall performance.
  arXiv  Detail & Related papers  (2021-12-14T06:19:05Z)
• Localized Calibration: Metrics and Recalibration [133.07044916594361]
  We propose a fine-grained calibration metric that spans the gap between fully global and fully individualized calibration. We then introduce a localized recalibration method, LoRe, that improves the LCE better than existing recalibration methods.
  arXiv  Detail & Related papers  (2021-02-22T07:22:12Z)
• Post-hoc Calibration of Neural Networks by g-Layers [51.42640515410253]
  In recent years, there is a surge of research on neural network calibration. It is known that minimizing Negative Log-Likelihood (NLL) will lead to a calibrated network on the training set if the global optimum is attained. We prove that even though the base network ($f$) does not lead to the global optimum of NLL, by adding additional layers ($g$) and minimizing NLL by optimizing the parameters of $g$ one can obtain a calibrated network.
  arXiv  Detail & Related papers  (2020-06-23T07:55:10Z)
• Calibration of Neural Networks using Splines [51.42640515410253]
  Measuring calibration error amounts to comparing two empirical distributions. We introduce a binning-free calibration measure inspired by the classical Kolmogorov-Smirnov (KS) statistical test. Our method consistently outperforms existing methods on KS error as well as other commonly used calibration measures.
  arXiv  Detail & Related papers  (2020-06-23T07:18:05Z)
• Calibrating Deep Neural Networks using Focal Loss [77.92765139898906]
  Miscalibration is a mismatch between a model's confidence and its correctness. We show that focal loss allows us to learn models that are already very well calibrated. We show that our approach achieves state-of-the-art calibration without compromising on accuracy in almost all cases.
  arXiv  Detail & Related papers  (2020-02-21T17:35:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.