Unimodal-Concentrated Loss: Fully Adaptive Label Distribution Learning for Ordinal Regression

• URL: http://arxiv.org/abs/2204.00309v1
• Date: Fri, 1 Apr 2022 09:40:11 GMT
• Title: Unimodal-Concentrated Loss: Fully Adaptive Label Distribution Learning for Ordinal Regression
• Authors: Qiang Li, Jingjing Wang, Zhaoliang Yao, Yachun Li, Pengju Yang, Jingwei Yan, Chunmao Wang, Shiliang Pu
• Abstract summary: We argue that existing ALDL algorithms do not fully exploit the intrinsic properties of ordinal regression. We propose a novel loss function for fully adaptive label distribution learning, namely unimodal-concentrated loss.
• Score: 32.35098925000738
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning from a label distribution has achieved promising results on ordinal regression tasks such as facial age and head pose estimation wherein, the concept of adaptive label distribution learning (ALDL) has drawn lots of attention recently for its superiority in theory. However, compared with the methods assuming fixed form label distribution, ALDL methods have not achieved better performance. We argue that existing ALDL algorithms do not fully exploit the intrinsic properties of ordinal regression. In this paper, we emphatically summarize that learning an adaptive label distribution on ordinal regression tasks should follow three principles. First, the probability corresponding to the ground-truth should be the highest in label distribution. Second, the probabilities of neighboring labels should decrease with the increase of distance away from the ground-truth, i.e., the distribution is unimodal. Third, the label distribution should vary with samples changing, and even be distinct for different instances with the same label, due to the different levels of difficulty and ambiguity. Under the premise of these principles, we propose a novel loss function for fully adaptive label distribution learning, namely unimodal-concentrated loss. Specifically, the unimodal loss derived from the learning to rank strategy constrains the distribution to be unimodal. Furthermore, the estimation error and the variance of the predicted distribution for a specific sample are integrated into the proposed concentrated loss to make the predicted distribution maximize at the ground-truth and vary according to the predicting uncertainty. Extensive experimental results on typical ordinal regression tasks including age and head pose estimation, show the superiority of our proposed unimodal-concentrated loss compared with existing loss functions.

Related papers

• Theory-inspired Label Shift Adaptation via Aligned Distribution Mixture [21.494268411607766]
  We propose an innovative label shift framework named as Aligned Distribution Mixture (ADM) Within this framework, we enhance four typical label shift methods by introducing modifications to the classifier training process. Considering the distinctiveness of the proposed one-step approach, we develop an efficient bi-level optimization strategy.
  arXiv  Detail & Related papers  (2024-11-04T12:51:57Z)
• Rejection via Learning Density Ratios [50.91522897152437]
  Classification with rejection emerges as a learning paradigm which allows models to abstain from making predictions. We propose a different distributional perspective, where we seek to find an idealized data distribution which maximizes a pretrained model's performance. Our framework is tested empirically over clean and noisy datasets.
  arXiv  Detail & Related papers  (2024-05-29T01:32:17Z)
• Generating Unbiased Pseudo-labels via a Theoretically Guaranteed Chebyshev Constraint to Unify Semi-supervised Classification and Regression [57.17120203327993]
  threshold-to-pseudo label process (T2L) in classification uses confidence to determine the quality of label. In nature, regression also requires unbiased methods to generate high-quality labels. We propose a theoretically guaranteed constraint for generating unbiased labels based on Chebyshev's inequality.
  arXiv  Detail & Related papers  (2023-11-03T08:39:35Z)
• Robust Outlier Rejection for 3D Registration with Variational Bayes [70.98659381852787]
  We develop a novel variational non-local network-based outlier rejection framework for robust alignment. We propose a voting-based inlier searching strategy to cluster the high-quality hypothetical inliers for transformation estimation.
  arXiv  Detail & Related papers  (2023-04-04T03:48:56Z)
• Dist-PU: Positive-Unlabeled Learning from a Label Distribution Perspective [89.5370481649529]
  We propose a label distribution perspective for PU learning in this paper. Motivated by this, we propose to pursue the label distribution consistency between predicted and ground-truth label distributions. Experiments on three benchmark datasets validate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2022-12-06T07:38:29Z)
• Tackling Instance-Dependent Label Noise with Dynamic Distribution Calibration [18.59803726676361]
  Instance-dependent label noise is realistic but rather challenging, where the label-corruption process depends on instances directly. It causes a severe distribution shift between the distributions of training and test data, which impairs the generalization of trained models. In this paper, to address the distribution shift in learning with instance-dependent label noise, a dynamic distribution-calibration strategy is adopted.
  arXiv  Detail & Related papers  (2022-10-11T03:50:52Z)
• Fairness Transferability Subject to Bounded Distribution Shift [5.62716254065607]
  Given an algorithmic predictor that is "fair" on some source distribution, will it still be fair on an unknown target distribution that differs from the source within some bound? We study the transferability of statistical group fairness for machine learning predictors subject to bounded distribution shifts.
  arXiv  Detail & Related papers  (2022-05-31T22:16:44Z)
• Shift Happens: Adjusting Classifiers [2.8682942808330703]
  Minimizing expected loss measured by a proper scoring rule, such as Brier score or log-loss (cross-entropy), is a common objective while training a probabilistic classifier. We propose methods that transform all predictions to (re)equalize the average prediction and the class distribution. We demonstrate experimentally that, when in practice the class distribution is known only approximately, there is often still a reduction in loss depending on the amount of shift and the precision to which the class distribution is known.
  arXiv  Detail & Related papers  (2021-11-03T21:27:27Z)
• Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization [77.24152933825238]
  We show that for linear classification tasks we need stronger restrictions on the distribution shifts, or otherwise OOD generalization is impossible. We prove that a form of the information bottleneck constraint along with invariance helps address key failures when invariant features capture all the information about the label and also retains the existing success when they do not.
  arXiv  Detail & Related papers  (2021-06-11T20:42:27Z)
• Distributional Reinforcement Learning via Moment Matching [54.16108052278444]
  We formulate a method that learns a finite set of statistics from each return distribution via neural networks. Our method can be interpreted as implicitly matching all orders of moments between a return distribution and its Bellman target. Experiments on the suite of Atari games show that our method outperforms the standard distributional RL baselines.
  arXiv  Detail & Related papers  (2020-07-24T05:18:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.