Using the Naive Bayes as a discriminative classifier

• URL: http://arxiv.org/abs/2012.13572v3
• Date: Fri, 5 Mar 2021 16:15:28 GMT
• Title: Using the Naive Bayes as a discriminative classifier
• Authors: Elie Azeraf, Emmanuel Monfrini, Wojciech Pieczynski
• Abstract summary: For classification tasks, probabilistic models can be categorized into two disjoint classes: generative or discriminative. The recent Entropic Forward-Backward algorithm shows that the Hidden Markov Model, considered as a generative model, can also match the discriminative one's definition. We show that the Naive Bayes classifier can also match the discriminative classifier definition, so it can be used in either a generative or a discriminative way.
• Score: 6.939768185086753
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: For classification tasks, probabilistic models can be categorized into two disjoint classes: generative or discriminative. It depends on the posterior probability computation of the label $x$ given the observation $y$, $p(x | y)$. On the one hand, generative classifiers, like the Naive Bayes or the Hidden Markov Model (HMM), need the computation of the joint probability p(x,y), before using the Bayes rule to compute $p(x | y)$. On the other hand, discriminative classifiers compute $p(x | y)$ directly, regardless of the observations' law. They are intensively used nowadays, with models as Logistic Regression, Conditional Random Fields (CRF), and Artificial Neural Networks. However, the recent Entropic Forward-Backward algorithm shows that the HMM, considered as a generative model, can also match the discriminative one's definition. This example leads to question if it is the case for other generative models. In this paper, we show that the Naive Bayes classifier can also match the discriminative classifier definition, so it can be used in either a generative or a discriminative way. Moreover, this observation also discusses the notion of Generative-Discriminative pairs, linking, for example, Naive Bayes and Logistic Regression, or HMM and CRF. Related to this point, we show that the Logistic Regression can be viewed as a particular case of the Naive Bayes used in a discriminative way.

Related papers

• Mitigating Nonlinear Algorithmic Bias in Binary Classification [0.0]
  This paper proposes the use of causal modeling to detect and mitigate bias that is nonlinear in the protected attribute. We show that the probability of getting correctly classified as "low risk" is lowest among young people. Based on the fitted causal model, the debiased- probability estimates are computed, showing improved fairness with little impact on overall accuracy.
  arXiv  Detail & Related papers  (2023-12-09T01:26:22Z)
• Revisiting Discriminative vs. Generative Classifiers: Theory and Implications [37.98169487351508]
  This paper is inspired by the statistical efficiency of naive Bayes. We present a multiclass $mathcalH$-consistency bound framework and an explicit bound for logistic loss. Experiments on various pre-trained deep vision models show that naive Bayes consistently converges faster as the number of data increases.
  arXiv  Detail & Related papers  (2023-02-05T08:30:42Z)
• Two-sample test based on Self-Organizing Maps [68.8204255655161]
  Machine-learning classifiers can be leveraged as a two-sample statistical test. Self-Organizing Maps are a dimensionality reduction initially devised as a data visualization tool. But since their original purpose is visualization, they can also offer insights.
  arXiv  Detail & Related papers  (2022-12-17T21:35:47Z)
• Parametric Classification for Generalized Category Discovery: A Baseline Study [70.73212959385387]
  Generalized Category Discovery (GCD) aims to discover novel categories in unlabelled datasets using knowledge learned from labelled samples. We investigate the failure of parametric classifiers, verify the effectiveness of previous design choices when high-quality supervision is available, and identify unreliable pseudo-labels as a key problem. We propose a simple yet effective parametric classification method that benefits from entropy regularisation, achieves state-of-the-art performance on multiple GCD benchmarks and shows strong robustness to unknown class numbers.
  arXiv  Detail & Related papers  (2022-11-21T18:47:11Z)
• On the Identifiability and Estimation of Causal Location-Scale Noise Models [122.65417012597754]
  We study the class of location-scale or heteroscedastic noise models (LSNMs) We show the causal direction is identifiable up to some pathological cases. We propose two estimators for LSNMs: an estimator based on (non-linear) feature maps, and one based on neural networks.
  arXiv  Detail & Related papers  (2022-10-13T17:18:59Z)
• Asymptotic Statistical Analysis of $f$-divergence GAN [13.587087960403199]
  Generative Adversarial Networks (GANs) have achieved great success in data generation. We consider the statistical behavior of the general $f$-divergence formulation of GAN. The resulting estimation method is referred to as Adversarial Gradient Estimation (AGE)
  arXiv  Detail & Related papers  (2022-09-14T18:08:37Z)
• Reusing the Task-specific Classifier as a Discriminator: Discriminator-free Adversarial Domain Adaptation [55.27563366506407]
  We introduce a discriminator-free adversarial learning network (DALN) for unsupervised domain adaptation (UDA) DALN achieves explicit domain alignment and category distinguishment through a unified objective. DALN compares favorably against the existing state-of-the-art (SOTA) methods on a variety of public datasets.
  arXiv  Detail & Related papers  (2022-04-08T04:40:18Z)
• Deriving discriminative classifiers from generative models [6.939768185086753]
  We show how a generative classifier induced from a generative model can also be computed in a discriminative way from the same model. We illustrate the interest of the new discriminative way of computing classifiers in the Natural Language Processing (NLP) framework.
  arXiv  Detail & Related papers  (2022-01-03T19:18:25Z)
• Large scale analysis of generalization error in learning using margin based classification methods [2.436681150766912]
  We derive the expression for the generalization error of a family of large-margin classifiers in the limit of both sample size $n$ and dimension $p$. For two layer neural networks, we reproduce the recently developed double descent' phenomenology for several classification models.
  arXiv  Detail & Related papers  (2020-07-16T20:31:26Z)
• Learning from Aggregate Observations [82.44304647051243]
  We study the problem of learning from aggregate observations where supervision signals are given to sets of instances. We present a general probabilistic framework that accommodates a variety of aggregate observations. Simple maximum likelihood solutions can be applied to various differentiable models.
  arXiv  Detail & Related papers  (2020-04-14T06:18:50Z)
• Neural Bayes: A Generic Parameterization Method for Unsupervised Representation Learning [175.34232468746245]
  We introduce a parameterization method called Neural Bayes. It allows computing statistical quantities that are in general difficult to compute. We show two independent use cases for this parameterization.
  arXiv  Detail & Related papers  (2020-02-20T22:28:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.