Supervised learning with probabilistic morphisms and kernel mean embeddings

• URL: http://arxiv.org/abs/2305.06348v5
• Date: Wed, 15 Nov 2023 15:12:42 GMT
• Title: Supervised learning with probabilistic morphisms and kernel mean embeddings
• Authors: H\^ong V\^an L\^e
• Abstract summary: I propose a generative model of supervised learning that unifies two approaches to supervised learning. I address two measurability problems, which have been ignored in statistical learning theory. I present a variant of Vapnik-Stefanuyk's regularization method for solving ill-posed problems.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper I propose a generative model of supervised learning that unifies two approaches to supervised learning, using a concept of a correct loss function. Addressing two measurability problems, which have been ignored in statistical learning theory, I propose to use convergence in outer probability to characterize the consistency of a learning algorithm. Building upon these results, I extend a result due to Cucker-Smale, which addresses the learnability of a regression model, to the setting of a conditional probability estimation problem. Additionally, I present a variant of Vapnik-Stefanuyk's regularization method for solving stochastic ill-posed problems, and using it to prove the generalizability of overparameterized supervised learning models.

Related papers

• On Discriminative Probabilistic Modeling for Self-Supervised Representation Learning [85.75164588939185]
  We study the discriminative probabilistic modeling problem on a continuous domain for (multimodal) self-supervised representation learning. We conduct generalization error analysis to reveal the limitation of current InfoNCE-based contrastive loss for self-supervised representation learning.
  arXiv  Detail & Related papers  (2024-10-11T18:02:46Z)
• Probability Distribution Learning: A theoretical framework for Deep Learning [5.281849820329249]
  This paper introduces probability distribution (PD) learning, a novel theoretical learning framework. PD learning focuses on the underlying distribution probability, which is modeled as a decomposed variable within simplex. We introduce a standard loss function and the gradient structural control (GSC) algorithm, and demonstrate that by employing this function, the optima of fitting error can be approached by the norm and structural error.
  arXiv  Detail & Related papers  (2024-06-09T06:49:22Z)
• Learning Latent Graph Structures and their Uncertainty [63.95971478893842]
  Graph Neural Networks (GNNs) use relational information as an inductive bias to enhance the model's accuracy. As task-relevant relations might be unknown, graph structure learning approaches have been proposed to learn them while solving the downstream prediction task.
  arXiv  Detail & Related papers  (2024-05-30T10:49:22Z)
• Outlier-Robust Learning of Ising Models Under Dobrushin's Condition [57.89518300699042]
  We study the problem of learning Ising models satisfying Dobrushin's condition in the outlier-robust setting where a constant fraction of the samples are adversarially corrupted. Our main result is to provide the first computationally efficient robust learning algorithm for this problem with near-optimal error guarantees.
  arXiv  Detail & Related papers  (2021-02-03T18:00:57Z)
• Uniform Error and Posterior Variance Bounds for Gaussian Process Regression with Application to Safe Control [11.42419064387567]
  We present a novel uniform error bound using Lipschitz and an analysis of the posterior variance function for a large class of kernels. We show how these results can be used to guarantee safe control of an unknown dynamical system.
  arXiv  Detail & Related papers  (2021-01-13T20:06:30Z)
• Robust Unsupervised Learning via L-Statistic Minimization [38.49191945141759]
  We present a general approach to this problem focusing on unsupervised learning. The key assumption is that the perturbing distribution is characterized by larger losses relative to a given class of admissible models. We prove uniform convergence bounds with respect to the proposed criterion for several popular models in unsupervised learning.
  arXiv  Detail & Related papers  (2020-12-14T10:36:06Z)
• Variance based sensitivity analysis for Monte Carlo and importance sampling reliability assessment with Gaussian processes [0.0]
  We propose a methodology to quantify the sensitivity of the probability of failure estimator to two uncertainty sources. This analysis also enables to control the whole error associated to the failure probability estimate and thus provides an accuracy criterion on the estimation. The approach is proposed for both a Monte Carlo based method as well as an importance sampling based method, seeking to improve the estimation of rare event probabilities.
  arXiv  Detail & Related papers  (2020-11-30T17:06:28Z)
• Trust but Verify: Assigning Prediction Credibility by Counterfactual Constrained Learning [123.3472310767721]
  Prediction credibility measures are fundamental in statistics and machine learning. These measures should account for the wide variety of models used in practice. The framework developed in this work expresses the credibility as a risk-fit trade-off.
  arXiv  Detail & Related papers  (2020-11-24T19:52:38Z)
• A Contraction Approach to Model-based Reinforcement Learning [11.701145942745274]
  We analyze the error in the cumulative reward using a contraction approach. We prove that branched rollouts can reduce this error. In this case, we show that GAN-type learning has an advantage over Behavioral Cloning when its discriminator is well-trained.
  arXiv  Detail & Related papers  (2020-09-18T02:03:14Z)
• Accounting for Unobserved Confounding in Domain Generalization [107.0464488046289]
  This paper investigates the problem of learning robust, generalizable prediction models from a combination of datasets. Part of the challenge of learning robust models lies in the influence of unobserved confounders. We demonstrate the empirical performance of our approach on healthcare data from different modalities.
  arXiv  Detail & Related papers  (2020-07-21T08:18:06Z)
• Good Classifiers are Abundant in the Interpolating Regime [64.72044662855612]
  We develop a methodology to compute precisely the full distribution of test errors among interpolating classifiers. We find that test errors tend to concentrate around a small typical value $varepsilon*$, which deviates substantially from the test error of worst-case interpolating model. Our results show that the usual style of analysis in statistical learning theory may not be fine-grained enough to capture the good generalization performance observed in practice.
  arXiv  Detail & Related papers  (2020-06-22T21:12:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.