Adversarial Surrogate Risk Bounds for Binary Classification

• URL: http://arxiv.org/abs/2506.09348v2
• Date: Tue, 07 Oct 2025 22:02:43 GMT
• Title: Adversarial Surrogate Risk Bounds for Binary Classification
• Authors: Natalie S. Frank,
• Abstract summary: A central concern in classification is the vulnerability of machine learning models to adversarial attacks.<n>Recent work has characterized the conditions under which any sequence minimizing the adversarial surrogate risk also minimizes the adversarial classification risk in the binary setting.<n>This paper provides surrogate risk bounds that quantify that convergence rate.
• Score: 1.7767466724342065
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A central concern in classification is the vulnerability of machine learning models to adversarial attacks. Adversarial training is one of the most popular techniques for training robust classifiers, which involves minimizing an adversarial surrogate risk. Recent work has characterized the conditions under which any sequence minimizing the adversarial surrogate risk also minimizes the adversarial classification risk in the binary setting, a property known as adversarial consistency. However, these results do not address the rate at which the adversarial classification risk approaches its optimal value along such a sequence. This paper provides surrogate risk bounds that quantify that convergence rate.

Related papers

• Constrained Language Model Policy Optimization via Risk-aware Stepwise Alignment [49.2305683068875]
  We propose Risk-aware Stepwise Alignment (RSA), a novel alignment method that incorporates risk awareness into the policy optimization process.<n> RSA mitigates risks induced by excessive model shift away from a reference policy, and it explicitly suppresses low-probability yet high-impact harmful behaviors.<n> Experimental results demonstrate that our method achieves high levels of helpfulness while ensuring strong safety.
  arXiv  Detail & Related papers  (2025-12-30T14:38:02Z)
• Adversarial Consistency and the Uniqueness of the Adversarial Bayes Classifier [0.0]
  Minimizing an adversarial surrogate risk is a common technique for learning robust classifiers. We show that under reasonable distributional assumptions, a convex surrogate loss is statistically consistent for adversarial learning iff the adversarial Bayes classifier satisfies a certain notion of uniqueness.
  arXiv  Detail & Related papers  (2024-04-26T12:16:08Z)
• Generalization Properties of Adversarial Training for $\ell_0$-Bounded Adversarial Attacks [47.22918498465056]
  In this paper, we aim to theoretically characterize the performance of adversarial training for an important class of neural networks. Deriving a generalization in this setting has two main challenges.
  arXiv  Detail & Related papers  (2024-02-05T22:57:33Z)
• Learning with Complementary Labels Revisited: The Selected-Completely-at-Random Setting Is More Practical [66.57396042747706]
  Complementary-label learning is a weakly supervised learning problem. We propose a consistent approach that does not rely on the uniform distribution assumption. We find that complementary-label learning can be expressed as a set of negative-unlabeled binary classification problems.
  arXiv  Detail & Related papers  (2023-11-27T02:59:17Z)
• Outlier Robust Adversarial Training [57.06824365801612]
  We introduce Outlier Robust Adversarial Training (ORAT) in this work. ORAT is based on a bi-level optimization formulation of adversarial training with a robust rank-based loss function. We show that the learning objective of ORAT satisfies the $mathcalH$-consistency in binary classification, which establishes it as a proper surrogate to adversarial 0/1 loss.
  arXiv  Detail & Related papers  (2023-09-10T21:36:38Z)
• Non-Asymptotic Bounds for Adversarial Excess Risk under Misspecified Models [9.65010022854885]
  We show that adversarial risk is equivalent to the risk induced by a distributional adversarial attack under certain smoothness conditions. To evaluate the generalization performance of the adversarial estimator, we study the adversarial excess risk.
  arXiv  Detail & Related papers  (2023-09-02T00:51:19Z)
• Adversarial Training Should Be Cast as a Non-Zero-Sum Game [121.95628660889628]
  Two-player zero-sum paradigm of adversarial training has not engendered sufficient levels of robustness. We show that the commonly used surrogate-based relaxation used in adversarial training algorithms voids all guarantees on robustness. A novel non-zero-sum bilevel formulation of adversarial training yields a framework that matches and in some cases outperforms state-of-the-art attacks.
  arXiv  Detail & Related papers  (2023-06-19T16:00:48Z)
• The Adversarial Consistency of Surrogate Risks for Binary Classification [20.03511985572199]
  adversarial training seeks to minimize the expected $0$-$1$ loss when each example can be maliciously corrupted within a small ball. We give a simple and complete characterization of the set of surrogate loss functions that are consistent. Our results reveal that the class of adversarially consistent surrogates is substantially smaller than in the standard setting.
  arXiv  Detail & Related papers  (2023-05-17T05:27:40Z)
• Safe Deployment for Counterfactual Learning to Rank with Exposure-Based Risk Minimization [63.93275508300137]
  We introduce a novel risk-aware Counterfactual Learning To Rank method with theoretical guarantees for safe deployment. Our experimental results demonstrate the efficacy of our proposed method, which is effective at avoiding initial periods of bad performance when little data is available.
  arXiv  Detail & Related papers  (2023-04-26T15:54:23Z)
• The Consistency of Adversarial Training for Binary Classification [12.208787849155048]
  adversarial training involves minimizing a supremum-based surrogate risk. We characterize which supremum-based surrogates are consistent for distributions absolutely continuous with respect to Lebesgue measure in binary classification.
  arXiv  Detail & Related papers  (2022-06-18T03:37:43Z)
• Existence and Minimax Theorems for Adversarial Surrogate Risks in Binary Classification [16.626667055542086]
  Adversarial training is one of the most popular methods for training methods robust to adversarial attacks. We prove and existence, regularity, and minimax theorems for adversarial surrogate risks.
  arXiv  Detail & Related papers  (2022-06-18T03:29:49Z)
• Benign Overfitting in Adversarially Robust Linear Classification [91.42259226639837]
  "Benign overfitting", where classifiers memorize noisy training data yet still achieve a good generalization performance, has drawn great attention in the machine learning community. We show that benign overfitting indeed occurs in adversarial training, a principled approach to defend against adversarial examples.
  arXiv  Detail & Related papers  (2021-12-31T00:27:31Z)
• How does the Combined Risk Affect the Performance of Unsupervised Domain Adaptation Approaches? [33.65954640678556]
  Unsupervised domain adaptation (UDA) aims to train a target classifier with labeled samples from the source domain and unlabeled samples from the target domain. E-MixNet employs enhanced mixup, a generic vicinal distribution, on the labeled source samples and pseudo-labeled target samples to calculate a proxy of the combined risk.
  arXiv  Detail & Related papers  (2020-12-30T00:46:57Z)
• Learning to Separate Clusters of Adversarial Representations for Robust Adversarial Detection [50.03939695025513]
  We propose a new probabilistic adversarial detector motivated by a recently introduced non-robust feature. In this paper, we consider the non-robust features as a common property of adversarial examples, and we deduce it is possible to find a cluster in representation space corresponding to the property. This idea leads us to probability estimate distribution of adversarial representations in a separate cluster, and leverage the distribution for a likelihood based adversarial detector.
  arXiv  Detail & Related papers  (2020-12-07T07:21:18Z)
• Calibrated Surrogate Losses for Adversarially Robust Classification [92.37268323142307]
  We show that no convex surrogate loss is respect with respect to adversarial 0-1 loss when restricted to linear models. We also show that if the underlying distribution satisfies the Massart's noise condition, convex losses can also be calibrated in the adversarial setting.
  arXiv  Detail & Related papers  (2020-05-28T02:40:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.