Certified Robustness from Approximate Gaussian Mixture Structures in Pretrained Latent Spaces

• URL: http://arxiv.org/abs/2605.25352v1
• Date: Mon, 25 May 2026 02:12:58 GMT
• Title: Certified Robustness from Approximate Gaussian Mixture Structures in Pretrained Latent Spaces
• Abstract summary: Deep learning models are vulnerable to adversarial perturbations.<n> Empirical defenses can achieve strong robustness in practice, but lack formal guarantees.<n>We propose a framework for designing certifiably robust classifiers that leverage latent structure in data representations.
• Score: 42.97530488302465
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep learning models are vulnerable to adversarial perturbations, raising important concerns for safety-critical deployment. Empirical defenses can achieve strong robustness in practice, but lack formal guarantees, motivating the need for certifiably robust classifiers. While certified methods provide formal guarantees, they often yield overly conservative bounds due to their inability to exploit structure in complex data distributions. In this work, we propose a framework for designing certifiably robust classifiers that leverages latent structure in data representations. We first analyze the Gaussian mixture setting, deriving necessary and sufficient conditions for the existence of robust classifiers and constructing a classifier with a closed-form robustness certificate and generalization guarantees. Our main contribution is to show that exact structure is not required: we prove that if a pretrained encoder maps inputs to a latent distribution that is $\varepsilon$-close (in KL divergence) to a Gaussian mixture, then certified accuracy degrades gracefully, with an explicit bound relating robustness under the true and approximate distributions. This result enables the direct use of pretrained models without requiring exact distributional assumptions. Empirically, our method achieves state-of-the-art or competitive certified accuracy on CIFAR-10 and ImageNet, while maintaining strong clean performance and low computational overhead. Overall, our work establishes approximate latent structure as a practical and principled route to certifiable robustness.

Related papers

• Provable Robustness against Backdoor Attacks via the Primal-Dual Perspective on Differential Privacy [51.758416625168]
  Randomized smoothing is a powerful tool for certifying robustness to adversarial perturbations.<n>We propose a framework for certification of complex, composed mechanisms.<n>We provide a principled and general framework for using composite mechanisms to certify robustness under complex threat models.
  arXiv  Detail & Related papers  (2026-05-20T22:17:29Z)
• Beyond Red-Teaming: Formal Guarantees of LLM Guardrail Classifiers [1.9839136494100942]
  Guardrails defend production language models against harmful behavior, but they provide no formal guarantees.<n>We propose two constructions of such regions: SVD-aligned hyper-rectangles and Gaussian Mixture Models.<n>Applying this framework to three author-trained Guardrails on the toxicity domain, every hyper-rectangle configuration returns SAT.<n>While GPT-2 and Llama-3.1-8B maintain robust coverage of 90% and 80% across varying boundaries, BERT's safety guarantees prove uniquely volatile.
  arXiv  Detail & Related papers  (2026-05-11T17:41:38Z)
• Efficient Credal Prediction through Decalibration [27.17944683306638]
  We propose an efficient method for credal prediction that is grounded in the notion of relative likelihood.<n>We show that our method yields credal sets with strong performance across diverse tasks.<n> Notably, we demonstrate credal prediction on models such as TabPFN and CLIP.
  arXiv  Detail & Related papers  (2026-03-09T15:30:10Z)
• Mixing Classifiers to Alleviate the Accuracy-Robustness Trade-Off [8.169499497403102]
  We propose a theoretically motivated formulation that mixes the output probabilities of a standard neural network and a robust neural network. Our numerical experiments verify that the mixed classifier noticeably improves the accuracy-robustness trade-off.
  arXiv  Detail & Related papers  (2023-11-26T02:25:30Z)
• Certifying Ensembles: A General Certification Theory with S-Lipschitzness [128.2881318211724]
  Ensembling has shown to be beneficial for generalisation, uncertainty estimation, calibration, and mitigating the effects of concept drift. In this work, we generalise Lipschitz continuity by introducing S-Lipschitz classifiers, which we use to analyse the theoretical robustness of ensembles.
  arXiv  Detail & Related papers  (2023-04-25T17:50:45Z)
• Confidence-aware Training of Smoothed Classifiers for Certified Robustness [75.95332266383417]
  We use "accuracy under Gaussian noise" as an easy-to-compute proxy of adversarial robustness for an input. Our experiments show that the proposed method consistently exhibits improved certified robustness upon state-of-the-art training methods.
  arXiv  Detail & Related papers  (2022-12-18T03:57:12Z)
• Adversarial Training with Rectified Rejection [114.83821848791206]
  We propose to use true confidence (T-Con) as a certainty oracle, and learn to predict T-Con by rectifying confidence. We prove that under mild conditions, a rectified confidence (R-Con) rejector and a confidence rejector can be coupled to distinguish any wrongly classified input from correctly classified ones.
  arXiv  Detail & Related papers  (2021-05-31T08:24:53Z)
• Trust but Verify: Assigning Prediction Credibility by Counterfactual Constrained Learning [123.3472310767721]
  Prediction credibility measures are fundamental in statistics and machine learning. These measures should account for the wide variety of models used in practice. The framework developed in this work expresses the credibility as a risk-fit trade-off.
  arXiv  Detail & Related papers  (2020-11-24T19:52:38Z)
• Adversarial Robustness of Supervised Sparse Coding [34.94566482399662]
  We consider a model that involves learning a representation while at the same time giving a precise generalization bound and a robustness certificate. We focus on the hypothesis class obtained by combining a sparsity-promoting encoder coupled with a linear encoder. We provide a robustness certificate for end-to-end classification.
  arXiv  Detail & Related papers  (2020-10-22T22:05:21Z)
• Regularized Training and Tight Certification for Randomized Smoothed Classifier with Provable Robustness [15.38718018477333]
  We derive a new regularized risk, in which the regularizer can adaptively encourage the accuracy and robustness of the smoothed counterpart. We also design a new certification algorithm, which can leverage the regularization effect to provide tighter robustness lower bound that holds with high probability.
  arXiv  Detail & Related papers  (2020-02-17T20:54:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.