A Notion of Uniqueness for the Adversarial Bayes Classifier

• URL: http://arxiv.org/abs/2404.16956v2
• Date: Fri, 17 May 2024 23:48:47 GMT
• Title: A Notion of Uniqueness for the Adversarial Bayes Classifier
• Authors: Natalie S. Frank,
• Abstract summary: We propose a new notion of uniqueness for the adversarial Bayes classifier in the setting of binary classification. We show that as the perturbation radius increases, certain the regularity of adversarial Bayes classifiers improves.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a new notion of uniqueness for the adversarial Bayes classifier in the setting of binary classification. Analyzing this concept produces a simple procedure for computing all adversarial Bayes classifiers for a well-motivated family of one dimensional data distributions. This characterization is then leveraged to show that as the perturbation radius increases, certain the regularity of adversarial Bayes classifiers improves. Various examples demonstrate that the boundary of the adversarial Bayes classifier frequently lies near the boundary of the Bayes classifier.

Related papers

• Generative Classifiers Avoid Shortcut Solutions [84.23247217037134]
  Discriminative approaches to classification often learn shortcuts that hold in-distribution but fail under minor distribution shift.<n>We show that generative classifiers can avoid this issue by modeling all features, both core and spurious, instead of mainly spurious ones.<n>We find that diffusion-based and autorerimigressive generative classifiers achieve state-of-the-art performance on five standard image and text distribution shift benchmarks.
  arXiv  Detail & Related papers  (2025-12-31T18:31:46Z)
• Discriminative classification with generative features: bridging Naive Bayes and logistic regression [43.205900007057146]
  We introduce Smart Bayes, a new classification framework that bridges generative and discriminative modeling.<n>From the generative perspective, Smart Bayes relaxes the fixed unit weights of Naive Bayes by allowing data-driven coefficients on density-ratio features.<n>From a discriminative perspective, it constructs transformed inputs as marginal log-density ratios that explicitly quantify how much more likely each feature value is under one class.
  arXiv  Detail & Related papers  (2025-11-30T21:34:24Z)
• Classification Error Bound for Low Bayes Error Conditions in Machine Learning [50.25063912757367]
  We study the relationship between the error mismatch and the Kullback-Leibler divergence in machine learning. Motivated by recent observations of low model-based classification errors in many machine learning tasks, we propose a linear approximation of the classification error bound for low Bayes error conditions.
  arXiv  Detail & Related papers  (2025-01-27T11:57:21Z)
• Adversarial Consistency and the Uniqueness of the Adversarial Bayes Classifier [0.0]
  Minimizing an adversarial surrogate risk is a common technique for learning robust classifiers. We show that under reasonable distributional assumptions, a convex surrogate loss is statistically consistent for adversarial learning iff the adversarial Bayes classifier satisfies a certain notion of uniqueness.
  arXiv  Detail & Related papers  (2024-04-26T12:16:08Z)
• The Lipschitz-Variance-Margin Tradeoff for Enhanced Randomized Smoothing [85.85160896547698]
  Real-life applications of deep neural networks are hindered by their unsteady predictions when faced with noisy inputs and adversarial attacks. We show how to design an efficient classifier with a certified radius by relying on noise injection into the inputs. Our novel certification procedure allows us to use pre-trained models with randomized smoothing, effectively improving the current certification radius in a zero-shot manner.
  arXiv  Detail & Related papers  (2023-09-28T22:41:47Z)
• Clustering risk in Non-parametric Hidden Markov and I.I.D. Models [4.728478219127298]
  We show that clustering based on the Bayes classifier does not always match the optimal Bayes clusterer.<n>A key quantity emerges, capturing the fundamental difficulty of both classification and clustering tasks.
  arXiv  Detail & Related papers  (2023-09-21T16:31:04Z)
• Optimal partition of feature using Bayesian classifier [0.0]
  In Naive Bayes, certain features are called independent features as they have no conditional correlation or dependency when predicting a classification. We propose a novel technique called the Comonotone-Independence (CIBer) which is able to overcome the challenges posed by the Naive Bayes method.
  arXiv  Detail & Related papers  (2023-04-27T21:19:06Z)
• Is the Performance of My Deep Network Too Good to Be True? A Direct Approach to Estimating the Bayes Error in Binary Classification [86.32752788233913]
  In classification problems, the Bayes error can be used as a criterion to evaluate classifiers with state-of-the-art performance. We propose a simple and direct Bayes error estimator, where we just take the mean of the labels that show emphuncertainty of the classes. Our flexible approach enables us to perform Bayes error estimation even for weakly supervised data.
  arXiv  Detail & Related papers  (2022-02-01T13:22:26Z)
• Benign Overfitting in Adversarially Robust Linear Classification [91.42259226639837]
  "Benign overfitting", where classifiers memorize noisy training data yet still achieve a good generalization performance, has drawn great attention in the machine learning community. We show that benign overfitting indeed occurs in adversarial training, a principled approach to defend against adversarial examples.
  arXiv  Detail & Related papers  (2021-12-31T00:27:31Z)
• Improving usual Naive Bayes classifier performances with Neural Naive Bayes based models [6.939768185086753]
  This paper introduces the original Neural Naive Bayes, modeling the parameters of the classifier induced from the Naive Bayes with neural network functions. We also introduce new Neural Pooled Markov Chain models, alleviating the independence condition.
  arXiv  Detail & Related papers  (2021-11-14T10:42:26Z)
• A Normative Model of Classifier Fusion [4.111899441919164]
  We present a hierarchical Bayesian model of probabilistic classification fusion based on a new correlated Dirichlet distribution. The proposed model naturally accommodates the classic Independent Opinion Pool and other independent fusion algorithms as special cases. It is evaluated by uncertainty reduction and correctness of fusion on synthetic and real-world data sets.
  arXiv  Detail & Related papers  (2021-06-03T11:52:13Z)
• Toward Scalable and Unified Example-based Explanation and Outlier Detection [128.23117182137418]
  We argue for a broader adoption of prototype-based student networks capable of providing an example-based explanation for their prediction. We show that our prototype-based networks beyond similarity kernels deliver meaningful explanations and promising outlier detection results without compromising classification accuracy.
  arXiv  Detail & Related papers  (2020-11-11T05:58:17Z)
• Learning and Evaluating Representations for Deep One-class Classification [59.095144932794646]
  We present a two-stage framework for deep one-class classification. We first learn self-supervised representations from one-class data, and then build one-class classifiers on learned representations. In experiments, we demonstrate state-of-the-art performance on visual domain one-class classification benchmarks.
  arXiv  Detail & Related papers  (2020-11-04T23:33:41Z)
• BaCOUn: Bayesian Classifers with Out-of-Distribution Uncertainty [23.100727871427367]
  We propose a Bayesian framework to obtain reliable uncertainty estimates for deep classifiers. Our approach consists of a plug-in "generator" used to augment the data with an additional class of points that lie on the boundary of the training data, followed by Bayesian inference on top of features that are trained to distinguish these "out-of-distribution" points.
  arXiv  Detail & Related papers  (2020-07-12T20:52:55Z)
• Classifier-independent Lower-Bounds for Adversarial Robustness [13.247278149124757]
  We theoretically analyse the limits of robustness to test-time adversarial and noisy examples in classification. We use optimal transport theory to derive variational formulae for the Bayes-optimal error a classifier can make on a given classification problem. We derive explicit lower-bounds on the Bayes-optimal error in the case of the popular distance-based attacks.
  arXiv  Detail & Related papers  (2020-06-17T16:46:39Z)
• Open-Set Recognition with Gaussian Mixture Variational Autoencoders [91.3247063132127]
  In inference, open-set classification is to either classify a sample into a known class from training or reject it as an unknown class. We train our model to cooperatively learn reconstruction and perform class-based clustering in the latent space. Our model achieves more accurate and robust open-set classification results, with an average F1 improvement of 29.5%.
  arXiv  Detail & Related papers  (2020-06-03T01:15:19Z)
• Bayesian Deep Learning and a Probabilistic Perspective of Generalization [56.69671152009899]
  We show that deep ensembles provide an effective mechanism for approximate Bayesian marginalization. We also propose a related approach that further improves the predictive distribution by marginalizing within basins of attraction.
  arXiv  Detail & Related papers  (2020-02-20T15:13:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.