Variational Imbalanced Regression: Fair Uncertainty Quantification via Probabilistic Smoothing

• URL: http://arxiv.org/abs/2306.06599v8
• Date: Sun, 10 Nov 2024 08:36:05 GMT
• Title: Variational Imbalanced Regression: Fair Uncertainty Quantification via Probabilistic Smoothing
• Authors: Ziyan Wang, Hao Wang,
• Abstract summary: Existing regression models tend to fall short in both accuracy and uncertainty estimation when the label distribution is imbalanced. We propose a probabilistic deep learning model, dubbed variational imbalanced regression (VIR) VIR performs well in imbalanced regression but naturally produces reasonable uncertainty estimation as a byproduct.
• Score: 11.291393872745951
• License:
• Abstract: Existing regression models tend to fall short in both accuracy and uncertainty estimation when the label distribution is imbalanced. In this paper, we propose a probabilistic deep learning model, dubbed variational imbalanced regression (VIR), which not only performs well in imbalanced regression but naturally produces reasonable uncertainty estimation as a byproduct. Different from typical variational autoencoders assuming I.I.D. representations (a data point's representation is not directly affected by other data points), our VIR borrows data with similar regression labels to compute the latent representation's variational distribution; furthermore, different from deterministic regression models producing point estimates, VIR predicts the entire normal-inverse-gamma distributions and modulates the associated conjugate distributions to impose probabilistic reweighting on the imbalanced data, thereby providing better uncertainty estimation. Experiments in several real-world datasets show that our VIR can outperform state-of-the-art imbalanced regression models in terms of both accuracy and uncertainty estimation. Code will soon be available at https://github.com/Wang-ML-Lab/variational-imbalanced-regression.

Related papers

• Risk and cross validation in ridge regression with correlated samples [72.59731158970894]
  We provide training examples for the in- and out-of-sample risks of ridge regression when the data points have arbitrary correlations. We further extend our analysis to the case where the test point has non-trivial correlations with the training set, setting often encountered in time series forecasting. We validate our theory across a variety of high dimensional data.
  arXiv  Detail & Related papers  (2024-08-08T17:27:29Z)
• Beyond the Norms: Detecting Prediction Errors in Regression Models [26.178065248948773]
  This paper tackles the challenge of detecting unreliable behavior in regression algorithms. We introduce the notion of unreliability in regression, when the output of the regressor exceeds a specified discrepancy (or error) We show empirical improvements in error detection for multiple regression tasks, consistently outperforming popular baseline approaches.
  arXiv  Detail & Related papers  (2024-06-11T05:51:44Z)
• High-dimensional analysis of ridge regression for non-identically distributed data with a variance profile [0.0]
  We study the predictive risk of the ridge estimator for linear regression with a variance profile. For certain class of variance profile, our work highlights the emergence of the well-known double descent phenomenon. We also investigate the similarities and differences that exist with the standard setting of independent and identically distributed data.
  arXiv  Detail & Related papers  (2024-03-29T14:24:49Z)
• Selective Nonparametric Regression via Testing [54.20569354303575]
  We develop an abstention procedure via testing the hypothesis on the value of the conditional variance at a given point. Unlike existing methods, the proposed one allows to account not only for the value of the variance itself but also for the uncertainty of the corresponding variance predictor.
  arXiv  Detail & Related papers  (2023-09-28T13:04:11Z)
• Uncertainty Voting Ensemble for Imbalanced Deep Regression [20.176217123752465]
  In this paper, we introduce UVOTE, a method for learning from imbalanced data. We replace traditional regression losses with negative log-likelihood, which also predicts sample-wise aleatoric uncertainty. We show that UVOTE consistently outperforms the prior art, while at the same time producing better-calibrated uncertainty estimates.
  arXiv  Detail & Related papers  (2023-05-24T14:12:21Z)
• Learning Probabilistic Ordinal Embeddings for Uncertainty-Aware Regression [91.3373131262391]
  Uncertainty is the only certainty there is. Traditionally, the direct regression formulation is considered and the uncertainty is modeled by modifying the output space to a certain family of probabilistic distributions. How to model the uncertainty within the present-day technologies for regression remains an open issue.
  arXiv  Detail & Related papers  (2021-03-25T06:56:09Z)
• Unlabelled Data Improves Bayesian Uncertainty Calibration under Covariate Shift [100.52588638477862]
  We develop an approximate Bayesian inference scheme based on posterior regularisation. We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations.
  arXiv  Detail & Related papers  (2020-06-26T13:50:19Z)
• Stable Prediction via Leveraging Seed Variable [73.9770220107874]
  Previous machine learning methods might exploit subtly spurious correlations in training data induced by non-causal variables for prediction. We propose a conditional independence test based algorithm to separate causal variables with a seed variable as priori, and adopt them for stable prediction. Our algorithm outperforms state-of-the-art methods for stable prediction.
  arXiv  Detail & Related papers  (2020-06-09T06:56:31Z)
• Balance-Subsampled Stable Prediction [55.13512328954456]
  We propose a novel balance-subsampled stable prediction (BSSP) algorithm based on the theory of fractional factorial design. A design-theoretic analysis shows that the proposed method can reduce the confounding effects among predictors induced by the distribution shift. Numerical experiments on both synthetic and real-world data sets demonstrate that our BSSP algorithm significantly outperforms the baseline methods for stable prediction across unknown test data.
  arXiv  Detail & Related papers  (2020-06-08T07:01:38Z)
• A Locally Adaptive Interpretable Regression [7.4267694612331905]
  Linear regression is one of the most interpretable prediction models. In this work, we introduce a locally adaptive interpretable regression (LoAIR) Our model achieves comparable or better predictive performance than the other state-of-the-art baselines.
  arXiv  Detail & Related papers  (2020-05-07T09:26:14Z)
• Stable Prediction with Model Misspecification and Agnostic Distribution Shift [41.26323389341987]
  In machine learning algorithms, two main assumptions are required to guarantee performance. One is that the test data are drawn from the same distribution as the training data, and the other is that the model is correctly specified. Under model misspecification, distribution shift between training and test data leads to inaccuracy of parameter estimation and instability of prediction across unknown test data. We propose a novel Decorrelated Weighting Regression (DWR) algorithm which jointly optimize a variable decorrelation regularizer and a weighted regression model.
  arXiv  Detail & Related papers  (2020-01-31T08:56:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.