LipNeXt: Scaling up Lipschitz-based Certified Robustness to Billion-parameter Models

• URL: http://arxiv.org/abs/2601.18513v1
• Date: Mon, 26 Jan 2026 14:18:55 GMT
• Title: LipNeXt: Scaling up Lipschitz-based Certified Robustness to Billion-parameter Models
• Authors: Kai Hu, Haoqi Hu, Matt Fredrikson,
• Abstract summary: Lipschitz-based certification offers efficient, deterministic robustness guarantees but has struggled to scale in model size, training efficiency, and ImageNet performance.<n>We introduce emphLipNeXt, the first emphconstraint-free and emphconvolution-free 1-Lipschitz architecture for certified robustness.
• Score: 15.634238824408989
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Lipschitz-based certification offers efficient, deterministic robustness guarantees but has struggled to scale in model size, training efficiency, and ImageNet performance. We introduce \emph{LipNeXt}, the first \emph{constraint-free} and \emph{convolution-free} 1-Lipschitz architecture for certified robustness. LipNeXt is built using two techniques: (1) a manifold optimization procedure that updates parameters directly on the orthogonal manifold and (2) a \emph{Spatial Shift Module} to model spatial pattern without convolutions. The full network uses orthogonal projections, spatial shifts, a simple 1-Lipschitz $β$-Abs nonlinearity, and $L_2$ spatial pooling to maintain tight Lipschitz control while enabling expressive feature mixing. Across CIFAR-10/100 and Tiny-ImageNet, LipNeXt achieves state-of-the-art clean and certified robust accuracy (CRA), and on ImageNet it scales to 1-2B large models, improving CRA over prior Lipschitz models (e.g., up to $+8\%$ at $\varepsilon{=}1$) while retaining efficient, stable low-precision training. These results demonstrate that Lipschitz-based certification can benefit from modern scaling trends without sacrificing determinism or efficiency.

Related papers

• SPLITZ: Certifiable Robustness via Split Lipschitz Randomized Smoothing [8.471466670802817]
  SPLITZ is a practical and novel approach to provide certifiable robustness to adversarial examples.<n> Motivation for SPLITZ comes from the observation that many standard deep networks exhibit heterogeneity in Lipschitz constants.<n>We show that SPLITZ consistently improves on existing state-of-the-art approaches in the MNIST, CIFAR-10 and ImageNet datasets.
  arXiv  Detail & Related papers  (2024-07-03T05:13:28Z)
• Certified Robust Models with Slack Control and Large Lipschitz Constants [102.69689641398227]
  We propose a Calibrated Lipschitz-Margin Loss (CLL) that addresses two problems. Firstly, commonly used margin losses do not adjust the penalties to the shrinking output distribution. Secondly, minimization of $K$ can lead to overly smooth decision functions. Our CLL addresses these issues by explicitly calibrating the loss w.r.t. margin and Lipschitz constant.
  arXiv  Detail & Related papers  (2023-09-12T12:23:49Z)
• Unlocking Deterministic Robustness Certification on ImageNet [39.439003787779434]
  This paper investigates strategies for expanding certifiably robust training to larger, deeper models. We show that fast ways of bounding the Lipschitz constant for conventional ResNets are loose, and show how to address this by designing a new residual block. We are able to scale up fast deterministic robustness guarantees to ImageNet, demonstrating that this approach to robust learning can be applied to real-world applications.
  arXiv  Detail & Related papers  (2023-01-29T21:40:04Z)
• Improved techniques for deterministic l2 robustness [63.34032156196848]
  Training convolutional neural networks (CNNs) with a strict 1-Lipschitz constraint under the $l_2$ norm is useful for adversarial robustness, interpretable gradients and stable training. We introduce a procedure to certify robustness of 1-Lipschitz CNNs by replacing the last linear layer with a 1-hidden layer. We significantly advance the state-of-the-art for standard and provable robust accuracies on CIFAR-10 and CIFAR-100.
  arXiv  Detail & Related papers  (2022-11-15T19:10:12Z)
• Efficiently Computing Local Lipschitz Constants of Neural Networks via Bound Propagation [79.13041340708395]
  Lipschitz constants are connected to many properties of neural networks, such as robustness, fairness, and generalization. Existing methods for computing Lipschitz constants either produce relatively loose upper bounds or are limited to small networks. We develop an efficient framework for computing the $ell_infty$ local Lipschitz constant of a neural network by tightly upper bounding the norm of Clarke Jacobian.
  arXiv  Detail & Related papers  (2022-10-13T22:23:22Z)
• Rethinking Lipschitz Neural Networks for Certified L-infinity Robustness [33.72713778392896]
  We study certified $ell_infty$ from a novel perspective of representing Boolean functions. We develop a unified Lipschitz network that generalizes prior works, and design a practical version that can be efficiently trained.
  arXiv  Detail & Related papers  (2022-10-04T17:55:27Z)
• Training Certifiably Robust Neural Networks with Efficient Local Lipschitz Bounds [99.23098204458336]
  Certified robustness is a desirable property for deep neural networks in safety-critical applications. We show that our method consistently outperforms state-of-the-art methods on MNIST and TinyNet datasets.
  arXiv  Detail & Related papers  (2021-11-02T06:44:10Z)
• Scalable Lipschitz Residual Networks with Convex Potential Flows [120.27516256281359]
  We show that using convex potentials in a residual network gradient flow provides a built-in $1$-Lipschitz transformation. A comprehensive set of experiments on CIFAR-10 demonstrates the scalability of our architecture and the benefit of our approach for $ell$ provable defenses.
  arXiv  Detail & Related papers  (2021-10-25T07:12:53Z)
• Robust Implicit Networks via Non-Euclidean Contractions [63.91638306025768]
  Implicit neural networks show improved accuracy and significant reduction in memory consumption. They can suffer from ill-posedness and convergence instability. This paper provides a new framework to design well-posed and robust implicit neural networks.
  arXiv  Detail & Related papers  (2021-06-06T18:05:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.