Balanced MSE for Imbalanced Visual Regression
- URL: http://arxiv.org/abs/2203.16427v1
- Date: Wed, 30 Mar 2022 16:21:42 GMT
- Title: Balanced MSE for Imbalanced Visual Regression
- Authors: Jiawei Ren, Mingyuan Zhang, Cunjun Yu, Ziwei Liu
- Abstract summary: Data imbalance exists ubiquitously in real-world visual regressions.
imbalanced regression focuses on continuous labels, which can be boundless and high-dimensional.
We propose a novel loss function, Balanced MSE, to accommodate the imbalanced training label distribution.
- Score: 36.97675494319161
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Data imbalance exists ubiquitously in real-world visual regressions, e.g.,
age estimation and pose estimation, hurting the model's generalizability and
fairness. Thus, imbalanced regression gains increasing research attention
recently. Compared to imbalanced classification, imbalanced regression focuses
on continuous labels, which can be boundless and high-dimensional and hence
more challenging. In this work, we identify that the widely used Mean Square
Error (MSE) loss function can be ineffective in imbalanced regression. We
revisit MSE from a statistical view and propose a novel loss function, Balanced
MSE, to accommodate the imbalanced training label distribution. We further
design multiple implementations of Balanced MSE to tackle different real-world
scenarios, particularly including the one that requires no prior knowledge
about the training label distribution. Moreover, to the best of our knowledge,
Balanced MSE is the first general solution to high-dimensional imbalanced
regression. Extensive experiments on both synthetic and three real-world
benchmarks demonstrate the effectiveness of Balanced MSE.
Related papers
- Imbalance in Regression Datasets [0.9374652839580183]
We argue that imbalance in regression is an equally important problem which has so far been overlooked.
Due to under- and over-representations in a data set's target distribution, regressors are prone to degenerate to naive models.
arXiv Detail & Related papers (2024-02-19T09:06:26Z) - Uncertainty-guided Boundary Learning for Imbalanced Social Event
Detection [64.4350027428928]
We propose a novel uncertainty-guided class imbalance learning framework for imbalanced social event detection tasks.
Our model significantly improves social event representation and classification tasks in almost all classes, especially those uncertain ones.
arXiv Detail & Related papers (2023-10-30T03:32:04Z) - A step towards understanding why classification helps regression [16.741816961905947]
We show that the effect of adding a classification loss is the most pronounced for regression with imbalanced data.
For a regression task, if the data sampling is imbalanced, then add a classification loss.
arXiv Detail & Related papers (2023-08-21T10:00:46Z) - Variational Imbalanced Regression: Fair Uncertainty Quantification via
Probabilistic Smoothing [13.339286071690394]
Existing regression models tend to fall short in both accuracy and uncertainty estimation when the label distribution is imbalanced.
We propose a probabilistic deep learning model, dubbed variational imbalanced regression (VIR)
VIR performs well in imbalanced regression but naturally produces reasonable uncertainty estimation as a byproduct.
arXiv Detail & Related papers (2023-06-11T06:27:06Z) - Mixture of Experts with Uncertainty Voting for Imbalanced Deep
Regression Problems [22.041067758144077]
We propose a mixture-of-experts approach to imbalanced regression problems.
We replace traditional regression losses with negative log-likelihood which also predicts sample-wise aleatoric uncertainty.
We show experimentally that such a loss handles the imbalance better.
arXiv Detail & Related papers (2023-05-24T14:12:21Z) - Theoretical Characterization of the Generalization Performance of
Overfitted Meta-Learning [70.52689048213398]
This paper studies the performance of overfitted meta-learning under a linear regression model with Gaussian features.
We find new and interesting properties that do not exist in single-task linear regression.
Our analysis suggests that benign overfitting is more significant and easier to observe when the noise and the diversity/fluctuation of the ground truth of each training task are large.
arXiv Detail & Related papers (2023-04-09T20:36:13Z) - An Embarrassingly Simple Baseline for Imbalanced Semi-Supervised
Learning [103.65758569417702]
Semi-supervised learning (SSL) has shown great promise in leveraging unlabeled data to improve model performance.
We consider a more realistic and challenging setting called imbalanced SSL, where imbalanced class distributions occur in both labeled and unlabeled data.
We study a simple yet overlooked baseline -- SimiS -- which tackles data imbalance by simply supplementing labeled data with pseudo-labels.
arXiv Detail & Related papers (2022-11-20T21:18:41Z) - Variation-Incentive Loss Re-weighting for Regression Analysis on Biased
Data [8.115323786541078]
We aim to improve the accuracy of the regression analysis by addressing the data skewness/bias during model training.
We propose a Variation-Incentive Loss re-weighting method (VILoss) to optimize the gradient descent-based model training for regression analysis.
arXiv Detail & Related papers (2021-09-14T10:22:21Z) - Regression Bugs Are In Your Model! Measuring, Reducing and Analyzing
Regressions In NLP Model Updates [68.09049111171862]
This work focuses on quantifying, reducing and analyzing regression errors in the NLP model updates.
We formulate the regression-free model updates into a constrained optimization problem.
We empirically analyze how model ensemble reduces regression.
arXiv Detail & Related papers (2021-05-07T03:33:00Z) - Supercharging Imbalanced Data Learning With Energy-based Contrastive
Representation Transfer [72.5190560787569]
In computer vision, learning from long tailed datasets is a recurring theme, especially for natural image datasets.
Our proposal posits a meta-distributional scenario, where the data generating mechanism is invariant across the label-conditional feature distributions.
This allows us to leverage a causal data inflation procedure to enlarge the representation of minority classes.
arXiv Detail & Related papers (2020-11-25T00:13:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.