Related papers: Certifying Ensembles: A General Certification Theory with S-Lipschitzness

Certifying Ensembles: A General Certification Theory with S-Lipschitzness

URL: http://arxiv.org/abs/2304.13019v1
Date: Tue, 25 Apr 2023 17:50:45 GMT
Title: Certifying Ensembles: A General Certification Theory with S-Lipschitzness
Authors: Aleksandar Petrov, Francisco Eiras, Amartya Sanyal, Philip H.S. Torr, Adel Bibi
Abstract summary: Ensembling has shown to be beneficial for generalisation, uncertainty estimation, calibration, and mitigating the effects of concept drift. In this work, we generalise Lipschitz continuity by introducing S-Lipschitz classifiers, which we use to analyse the theoretical robustness of ensembles.
Score: 128.2881318211724
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
Abstract: Improving and guaranteeing the robustness of deep learning models has been a topic of intense research. Ensembling, which combines several classifiers to provide a better model, has shown to be beneficial for generalisation, uncertainty estimation, calibration, and mitigating the effects of concept drift. However, the impact of ensembling on certified robustness is less well understood. In this work, we generalise Lipschitz continuity by introducing S-Lipschitz classifiers, which we use to analyse the theoretical robustness of ensembles. Our results are precise conditions when ensembles of robust classifiers are more robust than any constituent classifier, as well as conditions when they are less robust.

Related papers

Mixing Classifiers to Alleviate the Accuracy-Robustness Trade-Off [8.169499497403102]
We propose a theoretically motivated formulation that mixes the output probabilities of a standard neural network and a robust neural network. Our numerical experiments verify that the mixed classifier noticeably improves the accuracy-robustness trade-off.
arXiv Detail & Related papers (2023-11-26T02:25:30Z)
Uncertainty-guided Boundary Learning for Imbalanced Social Event Detection [64.4350027428928]
We propose a novel uncertainty-guided class imbalance learning framework for imbalanced social event detection tasks. Our model significantly improves social event representation and classification tasks in almost all classes, especially those uncertain ones.
arXiv Detail & Related papers (2023-10-30T03:32:04Z)
When Does Confidence-Based Cascade Deferral Suffice? [69.28314307469381]
Cascades are a classical strategy to enable inference cost to vary adaptively across samples. A deferral rule determines whether to invoke the next classifier in the sequence, or to terminate prediction. Despite being oblivious to the structure of the cascade, confidence-based deferral often works remarkably well in practice.
arXiv Detail & Related papers (2023-07-06T04:13:57Z)
Controlling the Complexity and Lipschitz Constant improves polynomial nets [55.121200972539114]
We derive new complexity bounds for the set of Coupled CP-Decomposition (CCP) and Nested Coupled CP-decomposition (NCP) models of Polynomial Nets. We propose a principled regularization scheme that we evaluate experimentally in six datasets and show that it improves the accuracy as well as the robustness of the models to adversarial perturbations.
arXiv Detail & Related papers (2022-02-10T14:54:29Z)
SmoothMix: Training Confidence-calibrated Smoothed Classifiers for Certified Robustness [61.212486108346695]
We propose a training scheme, coined SmoothMix, to control the robustness of smoothed classifiers via self-mixup. The proposed procedure effectively identifies over-confident, near off-class samples as a cause of limited robustness. Our experimental results demonstrate that the proposed method can significantly improve the certified $ell$-robustness of smoothed classifiers.
arXiv Detail & Related papers (2021-11-17T18:20:59Z)
On the Certified Robustness for Ensemble Models and Beyond [22.43134152931209]
Deep neural networks (DNN) are vulnerable to adversarial examples, which aim to mislead them. We analyze and provide the certified robustness for ensemble ML models. Inspired by the theoretical findings, we propose the lightweight Diversity Regularized Training (DRT) to train certifiably robust ensemble ML models.
arXiv Detail & Related papers (2021-07-22T18:10:41Z)
Adversarial Robustness of Supervised Sparse Coding [34.94566482399662]
We consider a model that involves learning a representation while at the same time giving a precise generalization bound and a robustness certificate. We focus on the hypothesis class obtained by combining a sparsity-promoting encoder coupled with a linear encoder. We provide a robustness certificate for end-to-end classification.
arXiv Detail & Related papers (2020-10-22T22:05:21Z)
Certified Distributional Robustness on Smoothed Classifiers [27.006844966157317]
We propose the worst-case adversarial loss over input distributions as a robustness certificate. By exploiting duality and the smoothness property, we provide an easy-to-compute upper bound as a surrogate for the certificate.
arXiv Detail & Related papers (2020-10-21T13:22:25Z)
Consistency Regularization for Certified Robustness of Smoothed Classifiers [89.72878906950208]
A recent technique of randomized smoothing has shown that the worst-case $ell$-robustness can be transformed into the average-case robustness. We found that the trade-off between accuracy and certified robustness of smoothed classifiers can be greatly controlled by simply regularizing the prediction consistency over noise.
arXiv Detail & Related papers (2020-06-07T06:57:43Z)
Generalised Lipschitz Regularisation Equals Distributional Robustness [47.44261811369141]
We give a very general equality result regarding the relationship between distributional robustness and regularisation. We show a new result explicating the connection between adversarial learning and distributional robustness.
arXiv Detail & Related papers (2020-02-11T04:19:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.