Nonparametric Probabilistic Regression with Coarse Learners

• URL: http://arxiv.org/abs/2210.16247v1
• Date: Fri, 28 Oct 2022 16:25:26 GMT
• Title: Nonparametric Probabilistic Regression with Coarse Learners
• Authors: Brian Lucena
• Abstract summary: We show that we can compute precise conditional densities with minimal assumptions on the shape or form of the density. We demonstrate this approach on a variety of datasets and show competitive performance, particularly on larger datasets.
• Score: 1.8275108630751844
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Probabilistic Regression refers to predicting a full probability density function for the target conditional on the features. We present a nonparametric approach to this problem which combines base classifiers (typically gradient boosted forests) trained on different coarsenings of the target value. By combining such classifiers and averaging the resulting densities, we are able to compute precise conditional densities with minimal assumptions on the shape or form of the density. We combine this approach with a structured cross-entropy loss function which serves to regularize and smooth the resulting densities. Prediction intervals computed from these densities are shown to have high fidelity in practice. Furthermore, examining the properties of these densities on particular observations can provide valuable insight. We demonstrate this approach on a variety of datasets and show competitive performance, particularly on larger datasets.

Related papers

• Generative modeling of density regression through tree flows [3.0262553206264893]
  We propose a flow-based generative model tailored for the density regression task on tabular data. We introduce a training algorithm for fitting the tree-based transforms using a divide-and-conquer strategy. Our method consistently achieves comparable or superior performance at a fraction of the training and sampling budget.
  arXiv  Detail & Related papers  (2024-06-07T21:07:35Z)
• Collaborative Heterogeneous Causal Inference Beyond Meta-analysis [68.4474531911361]
  We propose a collaborative inverse propensity score estimator for causal inference with heterogeneous data. Our method shows significant improvements over the methods based on meta-analysis when heterogeneity increases.
  arXiv  Detail & Related papers  (2024-04-24T09:04:36Z)
• Empirical Density Estimation based on Spline Quasi-Interpolation with applications to Copulas clustering modeling [0.0]
  Density estimation is a fundamental technique employed in various fields to model and to understand the underlying distribution of data. In this paper we propose the mono-variate approximation of the density using quasi-interpolation. The presented algorithm is validated on artificial and real datasets.
  arXiv  Detail & Related papers  (2024-02-18T11:49:38Z)
• Leveraging Self-Consistency for Data-Efficient Amortized Bayesian Inference [9.940560505044122]
  We propose a method to improve the efficiency and accuracy of amortized Bayesian inference. We estimate the marginal likelihood based on approximate representations of the joint model.
  arXiv  Detail & Related papers  (2023-10-06T17:41:41Z)
• Statistical Efficiency of Score Matching: The View from Isoperimetry [96.65637602827942]
  We show a tight connection between statistical efficiency of score matching and the isoperimetric properties of the distribution being estimated. We formalize these results both in the sample regime and in the finite regime.
  arXiv  Detail & Related papers  (2022-10-03T06:09:01Z)
• Learning Structured Gaussians to Approximate Deep Ensembles [10.055143995729415]
  This paper proposes using a sparse-structured multivariate Gaussian to provide a closed-form approxorimator for dense image prediction tasks. We capture the uncertainty and structured correlations in the predictions explicitly in a formal distribution, rather than implicitly through sampling alone. We demonstrate the merits of our approach on monocular depth estimation and show that the advantages of our approach are obtained with comparable quantitative performance.
  arXiv  Detail & Related papers  (2022-03-29T12:34:43Z)
• Featurized Density Ratio Estimation [82.40706152910292]
  In our work, we propose to leverage an invertible generative model to map the two distributions into a common feature space prior to estimation. This featurization brings the densities closer together in latent space, sidestepping pathological scenarios where the learned density ratios in input space can be arbitrarily inaccurate. At the same time, the invertibility of our feature map guarantees that the ratios computed in feature space are equivalent to those in input space.
  arXiv  Detail & Related papers  (2021-07-05T18:30:26Z)
• Nonparametric Score Estimators [49.42469547970041]
  Estimating the score from a set of samples generated by an unknown distribution is a fundamental task in inference and learning of probabilistic models. We provide a unifying view of these estimators under the framework of regularized nonparametric regression. We propose score estimators based on iterative regularization that enjoy computational benefits from curl-free kernels and fast convergence.
  arXiv  Detail & Related papers  (2020-05-20T15:01:03Z)
• Asymptotic Analysis of an Ensemble of Randomly Projected Linear Discriminants [94.46276668068327]
  In [1], an ensemble of randomly projected linear discriminants is used to classify datasets. We develop a consistent estimator of the misclassification probability as an alternative to the computationally-costly cross-validation estimator. We also demonstrate the use of our estimator for tuning the projection dimension on both real and synthetic data.
  arXiv  Detail & Related papers  (2020-04-17T12:47:04Z)
• Bayesian Deep Learning and a Probabilistic Perspective of Generalization [56.69671152009899]
  We show that deep ensembles provide an effective mechanism for approximate Bayesian marginalization. We also propose a related approach that further improves the predictive distribution by marginalizing within basins of attraction.
  arXiv  Detail & Related papers  (2020-02-20T15:13:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.