Adversarial Surrogate Risk Bounds for Binary Classification

• URL: http://arxiv.org/abs/2506.09348v1
• Date: Wed, 11 Jun 2025 02:57:08 GMT
• Title: Adversarial Surrogate Risk Bounds for Binary Classification
• Authors: Natalie S. Frank,
• Abstract summary: Adversarial training is one of the most popular techniques for training robust classifiers.<n>This paper provides surrogate risk bounds that quantify the convergence rate.<n>We derive distribution-dependent surrogate risk bounds in the standard (non-adversarial) learning setting.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A central concern in classification is the vulnerability of machine learning models to adversarial attacks. Adversarial training is one of the most popular techniques for training robust classifiers, which involves minimizing an adversarial surrogate risk. Recent work characterized when a minimizing sequence of an adversarial surrogate risk is also a minimizing sequence of the adversarial classification risk for binary classification -- a property known as adversarial consistency. However, these results do not address the rate at which the adversarial classification risk converges to its optimal value for such a sequence of functions that minimize the adversarial surrogate. This paper provides surrogate risk bounds that quantify that convergence rate. Additionally, we derive distribution-dependent surrogate risk bounds in the standard (non-adversarial) learning setting, that may be of independent interest.

Related papers

• Adversarial Consistency and the Uniqueness of the Adversarial Bayes Classifier [0.0]
  Minimizing an adversarial surrogate risk is a common technique for learning robust classifiers. We show that under reasonable distributional assumptions, a convex surrogate loss is statistically consistent for adversarial learning iff the adversarial Bayes classifier satisfies a certain notion of uniqueness.
  arXiv  Detail & Related papers  (2024-04-26T12:16:08Z)
• Generalization Properties of Adversarial Training for $\ell_0$-Bounded Adversarial Attacks [47.22918498465056]
  In this paper, we aim to theoretically characterize the performance of adversarial training for an important class of neural networks. Deriving a generalization in this setting has two main challenges.
  arXiv  Detail & Related papers  (2024-02-05T22:57:33Z)
• Learning with Complementary Labels Revisited: The Selected-Completely-at-Random Setting Is More Practical [66.57396042747706]
  Complementary-label learning is a weakly supervised learning problem. We propose a consistent approach that does not rely on the uniform distribution assumption. We find that complementary-label learning can be expressed as a set of negative-unlabeled binary classification problems.
  arXiv  Detail & Related papers  (2023-11-27T02:59:17Z)
• Adversarial Training Should Be Cast as a Non-Zero-Sum Game [121.95628660889628]
  Two-player zero-sum paradigm of adversarial training has not engendered sufficient levels of robustness. We show that the commonly used surrogate-based relaxation used in adversarial training algorithms voids all guarantees on robustness. A novel non-zero-sum bilevel formulation of adversarial training yields a framework that matches and in some cases outperforms state-of-the-art attacks.
  arXiv  Detail & Related papers  (2023-06-19T16:00:48Z)
• The Adversarial Consistency of Surrogate Risks for Binary Classification [20.03511985572199]
  adversarial training seeks to minimize the expected $0$-$1$ loss when each example can be maliciously corrupted within a small ball. We give a simple and complete characterization of the set of surrogate loss functions that are consistent. Our results reveal that the class of adversarially consistent surrogates is substantially smaller than in the standard setting.
  arXiv  Detail & Related papers  (2023-05-17T05:27:40Z)
• Safe Deployment for Counterfactual Learning to Rank with Exposure-Based Risk Minimization [63.93275508300137]
  We introduce a novel risk-aware Counterfactual Learning To Rank method with theoretical guarantees for safe deployment. Our experimental results demonstrate the efficacy of our proposed method, which is effective at avoiding initial periods of bad performance when little data is available.
  arXiv  Detail & Related papers  (2023-04-26T15:54:23Z)
• The Consistency of Adversarial Training for Binary Classification [12.208787849155048]
  adversarial training involves minimizing a supremum-based surrogate risk. We characterize which supremum-based surrogates are consistent for distributions absolutely continuous with respect to Lebesgue measure in binary classification.
  arXiv  Detail & Related papers  (2022-06-18T03:37:43Z)
• Existence and Minimax Theorems for Adversarial Surrogate Risks in Binary Classification [16.626667055542086]
  Adversarial training is one of the most popular methods for training methods robust to adversarial attacks. We prove and existence, regularity, and minimax theorems for adversarial surrogate risks.
  arXiv  Detail & Related papers  (2022-06-18T03:29:49Z)
• Benign Overfitting in Adversarially Robust Linear Classification [91.42259226639837]
  "Benign overfitting", where classifiers memorize noisy training data yet still achieve a good generalization performance, has drawn great attention in the machine learning community. We show that benign overfitting indeed occurs in adversarial training, a principled approach to defend against adversarial examples.
  arXiv  Detail & Related papers  (2021-12-31T00:27:31Z)
• How does the Combined Risk Affect the Performance of Unsupervised Domain Adaptation Approaches? [33.65954640678556]
  Unsupervised domain adaptation (UDA) aims to train a target classifier with labeled samples from the source domain and unlabeled samples from the target domain. E-MixNet employs enhanced mixup, a generic vicinal distribution, on the labeled source samples and pseudo-labeled target samples to calculate a proxy of the combined risk.
  arXiv  Detail & Related papers  (2020-12-30T00:46:57Z)
• Learning to Separate Clusters of Adversarial Representations for Robust Adversarial Detection [50.03939695025513]
  We propose a new probabilistic adversarial detector motivated by a recently introduced non-robust feature. In this paper, we consider the non-robust features as a common property of adversarial examples, and we deduce it is possible to find a cluster in representation space corresponding to the property. This idea leads us to probability estimate distribution of adversarial representations in a separate cluster, and leverage the distribution for a likelihood based adversarial detector.
  arXiv  Detail & Related papers  (2020-12-07T07:21:18Z)
• Calibrated Surrogate Losses for Adversarially Robust Classification [92.37268323142307]
  We show that no convex surrogate loss is respect with respect to adversarial 0-1 loss when restricted to linear models. We also show that if the underlying distribution satisfies the Massart's noise condition, convex losses can also be calibrated in the adversarial setting.
  arXiv  Detail & Related papers  (2020-05-28T02:40:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.