Beyond Homoscedasticity: Decoupled Uncertainty Optimization for Deep Imbalanced Regression

• URL: http://arxiv.org/abs/2609.04995v1
• Date: Fri, 04 Sep 2026 11:06:57 GMT
• Title: Beyond Homoscedasticity: Decoupled Uncertainty Optimization for Deep Imbalanced Regression
• Abstract summary: Deep Imbalanced Regression is pervasive in continuous prediction tasks across diverse modalities.<n>We propose DUO, an uncertainty-aware long-tailed regression framework.<n>We show that DUO achieves the best few-shot bMAE and GM on IMDB-WIKI-DIR, AgeDB-DIR, and AAV2-DIR while remaining competitive on few-shot MAE.
• Score: 7.011984503405774
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep Imbalanced Regression (DIR) is pervasive in continuous prediction tasks across diverse modalities, such as age estimation, depth prediction, and protein mutation activity prediction, where label-scarce tail samples often carry higher practical value. However, most existing methods still learn deterministic point mappings under mean squared error or its simple variants, implicitly assuming a uniform uncertainty level across all samples and thereby overlooking the instance-wise heteroscedasticity that is widespread in long-tailed data. We further point out that even heteroscedastic negative log-likelihood suffers from a gradient coupling issue, which, under DIR scenarios, weakens the learning signal of hard tail samples and leads to optimization inertia as well as tail underfitting. To address this, we propose DUO, an uncertainty-aware long-tailed regression framework. Specifically, the proposed method models the regression target as a conditional Gaussian distribution to explicitly characterize instance-level predictive uncertainty, and transforms uncertainty into a dynamic enhancement signal for tail samples through decoupled mean-variance optimization. Furthermore, we design a distribution-guided contrastive learning mechanism that adaptively constructs positive and negative pairs based on the overlap between sample distributions, thereby alleviating feature looseness and cross-label semantic entanglement. Across visual and biological DIR benchmarks, DUO achieves the best few-shot bMAE and GM on IMDB-WIKI-DIR, AgeDB-DIR, and AAV2-DIR while remaining competitive on few-shot MAE.

Related papers

• Augmented Inverse Hybrid Weighting: Robust Inference under Deterministic and Random Distribution Shifts [17.842727239557995]
  We introduce a new model that allows non-systematic changes between two population laws after systematic shifts are accounted for.<n>Such residual shift is modeled as random perturbations to the probability space that cannot be represented in a learnable way.<n>Under pure random perturbations, this principle yields Augmented Inverse Distance Weighting (AIDW), which uses regression augmentation and variance-optimal dataset-level pooling.<n>For mixed shifts, we develop Augmented Inverse Hybrid Weighting (AIHW), which interpolates between AIDW and standard augmented importance weighting.
  arXiv  Detail & Related papers  (2026-08-01T15:02:13Z)
• Joint Model and Data Sparsification via the Marginal Likelihood [53.29070892356214]
  We propose jointly learning individual feature and sample relevancies, enabling simultaneous model and data sparsification.<n>This symmetric pruning of model and data offers a natural extension that preserves conjugacy.<n> Empirical results across diverse regression tasks affirm that a joint ARD approach consistently yields both sparse and robust prediction models.
  arXiv  Detail & Related papers  (2026-05-28T13:26:53Z)
• Amortized Variational Inference for Logistic Regression with Missing Covariates [4.027766797152636]
  Amortized Variational Inference for Logistic Regression (AV-LR) is a unified end-to-end framework for binary logistic regression.<n>AV-LR integrates a probabilistic generative model with a simple amortized inference network, trained jointly by maximizing the evidence lower bound.<n>It achieves estimation accuracy comparable to or better than state-of-the-art EM-like algorithms, with significantly lower computational cost.
  arXiv  Detail & Related papers  (2026-03-22T14:00:59Z)
• Risk and cross validation in ridge regression with correlated samples [72.59731158970894]
  We provide training examples for the in- and out-of-sample risks of ridge regression when the data points have arbitrary correlations.<n>We demonstrate that in this setting, the generalized cross validation estimator (GCV) fails to correctly predict the out-of-sample risk.<n>We further extend our analysis to the case where the test point has nontrivial correlations with the training set, a setting often encountered in time series forecasting.
  arXiv  Detail & Related papers  (2024-08-08T17:27:29Z)
• ROTI-GCV: Generalized Cross-Validation for right-ROTationally Invariant Data [5.830619388189558]
  Two key tasks in high-dimensional regularized regression are tuning the regularization strength for accurate predictions and estimating the out-of-sample risk.<n>We introduce a new framework, ROTI-GCV, for reliably performing cross-validation under challenging conditions.
  arXiv  Detail & Related papers  (2024-06-17T15:50:00Z)
• Fully Heteroscedastic Count Regression with Deep Double Poisson Networks [6.976150282812484]
  Deep Double Poisson Network (DDPN) is a novel neural discrete count regression model.<n>We show that DDPN exhibits robust regression properties similar to heteroscedastic Gaussian models.<n>Experiments on diverse datasets demonstrate that DDPN outperforms current baselines in accuracy, calibration, and out-of-distribution detection.
  arXiv  Detail & Related papers  (2024-06-13T16:02:03Z)
• Quantifying predictive uncertainty of aphasia severity in stroke patients with sparse heteroscedastic Bayesian high-dimensional regression [47.1405366895538]
  Sparse linear regression methods for high-dimensional data commonly assume that residuals have constant variance, which can be violated in practice. This paper proposes estimating high-dimensional heteroscedastic linear regression models using a heteroscedastic partitioned empirical Bayes Expectation Conditional Maximization algorithm.
  arXiv  Detail & Related papers  (2023-09-15T22:06:29Z)
• Variational Imbalanced Regression: Fair Uncertainty Quantification via Probabilistic Smoothing [11.291393872745951]
  Existing regression models tend to fall short in both accuracy and uncertainty estimation when the label distribution is imbalanced. We propose a probabilistic deep learning model, dubbed variational imbalanced regression (VIR) VIR performs well in imbalanced regression but naturally produces reasonable uncertainty estimation as a byproduct.
  arXiv  Detail & Related papers  (2023-06-11T06:27:06Z)
• Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness [61.827054365139645]
  Variational Autoencoder (VAE) approximates the posterior of latent variables based on amortized variational inference. We propose an alternative model, DU-VAE, for learning a more Diverse and less Uncertain latent space.
  arXiv  Detail & Related papers  (2021-10-24T07:58:13Z)
• On the Double Descent of Random Features Models Trained with SGD [78.0918823643911]
  We study properties of random features (RF) regression in high dimensions optimized by gradient descent (SGD) We derive precise non-asymptotic error bounds of RF regression under both constant and adaptive step-size SGD setting. We observe the double descent phenomenon both theoretically and empirically.
  arXiv  Detail & Related papers  (2021-10-13T17:47:39Z)
• Multivariate Probabilistic Regression with Natural Gradient Boosting [63.58097881421937]
  We propose a Natural Gradient Boosting (NGBoost) approach based on nonparametrically modeling the conditional parameters of the multivariate predictive distribution. Our method is robust, works out-of-the-box without extensive tuning, is modular with respect to the assumed target distribution, and performs competitively in comparison to existing approaches.
  arXiv  Detail & Related papers  (2021-06-07T17:44:49Z)
• Unlabelled Data Improves Bayesian Uncertainty Calibration under Covariate Shift [100.52588638477862]
  We develop an approximate Bayesian inference scheme based on posterior regularisation. We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations.
  arXiv  Detail & Related papers  (2020-06-26T13:50:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.