Generative vs. Discriminative modeling under the lens of uncertainty quantification

• URL: http://arxiv.org/abs/2406.09172v1
• Date: Thu, 13 Jun 2024 14:32:43 GMT
• Title: Generative vs. Discriminative modeling under the lens of uncertainty quantification
• Authors: Elouan Argouarc'h, François Desbouvries, Eric Barat, Eiji Kawasaki,
• Abstract summary: In this paper, we undertake a comparative analysis of generative and discriminative approaches. We compare the ability of both approaches to leverage information from various sources in an uncertainty aware inference. We propose a general sampling scheme enabling supervised learning for both approaches, as well as semi-supervised learning when compatible with the considered modeling approach.
• Score: 0.929965561686354
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Learning a parametric model from a given dataset indeed enables to capture intrinsic dependencies between random variables via a parametric conditional probability distribution and in turn predict the value of a label variable given observed variables. In this paper, we undertake a comparative analysis of generative and discriminative approaches which differ in their construction and the structure of the underlying inference problem. Our objective is to compare the ability of both approaches to leverage information from various sources in an epistemic uncertainty aware inference via the posterior predictive distribution. We assess the role of a prior distribution, explicit in the generative case and implicit in the discriminative case, leading to a discussion about discriminative models suffering from imbalanced dataset. We next examine the double role played by the observed variables in the generative case, and discuss the compatibility of both approaches with semi-supervised learning. We also provide with practical insights and we examine how the modeling choice impacts the sampling from the posterior predictive distribution. With regard to this, we propose a general sampling scheme enabling supervised learning for both approaches, as well as semi-supervised learning when compatible with the considered modeling approach. Throughout this paper, we illustrate our arguments and conclusions using the example of affine regression, and validate our comparative analysis through classification simulations using neural network based models.

Related papers

• Aggregation Weighting of Federated Learning via Generalization Bound Estimation [65.8630966842025]
  Federated Learning (FL) typically aggregates client model parameters using a weighting approach determined by sample proportions. We replace the aforementioned weighting method with a new strategy that considers the generalization bounds of each local model.
  arXiv  Detail & Related papers  (2023-11-10T08:50:28Z)
• Bayesian Additive Main Effects and Multiplicative Interaction Models using Tensor Regression for Multi-environmental Trials [0.0]
  We propose a Bayesian tensor regression model to accommodate the effect of multiple factors on phenotype prediction. We adopt a set of prior distributions that resolve identifiability issues that may arise between the parameters in the model. We explore the applicability of our model by analysing real-world data related to wheat production across Ireland from 2010 to 2019.
  arXiv  Detail & Related papers  (2023-01-09T19:54:50Z)
• Learning Invariant Representations under General Interventions on the Response [2.725698729450241]
  We focus on linear structural causal models (SCMs) and introduce invariant matching property (IMP) We analyze the generalization errors of our method under both the discrete and continuous environment settings.
  arXiv  Detail & Related papers  (2022-08-22T03:09:17Z)
• Instance-Based Neural Dependency Parsing [56.63500180843504]
  We develop neural models that possess an interpretable inference process for dependency parsing. Our models adopt instance-based inference, where dependency edges are extracted and labeled by comparing them to edges in a training set.
  arXiv  Detail & Related papers  (2021-09-28T05:30:52Z)
• MINIMALIST: Mutual INformatIon Maximization for Amortized Likelihood Inference from Sampled Trajectories [61.3299263929289]
  Simulation-based inference enables learning the parameters of a model even when its likelihood cannot be computed in practice. One class of methods uses data simulated with different parameters to infer an amortized estimator for the likelihood-to-evidence ratio. We show that this approach can be formulated in terms of mutual information between model parameters and simulated data.
  arXiv  Detail & Related papers  (2021-06-03T12:59:16Z)
• Latent Causal Invariant Model [128.7508609492542]
  Current supervised learning can learn spurious correlation during the data-fitting process. We propose a Latent Causal Invariance Model (LaCIM) which pursues causal prediction.
  arXiv  Detail & Related papers  (2020-11-04T10:00:27Z)
• Accounting for Unobserved Confounding in Domain Generalization [107.0464488046289]
  This paper investigates the problem of learning robust, generalizable prediction models from a combination of datasets. Part of the challenge of learning robust models lies in the influence of unobserved confounders. We demonstrate the empirical performance of our approach on healthcare data from different modalities.
  arXiv  Detail & Related papers  (2020-07-21T08:18:06Z)
• Achieving Equalized Odds by Resampling Sensitive Attributes [13.114114427206678]
  We present a flexible framework for learning predictive models that approximately satisfy the equalized odds notion of fairness. This differentiable functional is used as a penalty driving the model parameters towards equalized odds. We develop a formal hypothesis test to detect whether a prediction rule violates this property, the first such test in the literature.
  arXiv  Detail & Related papers  (2020-06-08T00:18:34Z)
• Decision-Making with Auto-Encoding Variational Bayes [71.44735417472043]
  We show that a posterior approximation distinct from the variational distribution should be used for making decisions. Motivated by these theoretical results, we propose learning several approximate proposals for the best model. In addition to toy examples, we present a full-fledged case study of single-cell RNA sequencing.
  arXiv  Detail & Related papers  (2020-02-17T19:23:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.