RankSim: Ranking Similarity Regularization for Deep Imbalanced Regression

• URL: http://arxiv.org/abs/2205.15236v1
• Date: Mon, 30 May 2022 16:51:25 GMT
• Title: RankSim: Ranking Similarity Regularization for Deep Imbalanced Regression
• Abstract summary: RankSim encodes an inductive bias that samples closer in label space should also be closer in feature space. RankSim is complementary to conventional imbalanced learning techniques.
• Score: 48.58405556678924
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Data imbalance, in which a plurality of the data samples come from a small proportion of labels, poses a challenge in training deep neural networks. Unlike classification, in regression the labels are continuous, potentially boundless, and form a natural ordering. These distinct features of regression call for new techniques that leverage the additional information encoded in label-space relationships. This paper presents the RankSim (ranking similarity) regularizer for deep imbalanced regression, which encodes an inductive bias that samples that are closer in label space should also be closer in feature space. In contrast to recent distribution smoothing based approaches, RankSim captures both nearby and distant relationships: for a given data sample, RankSim encourages the sorted list of its neighbors in label space to match the sorted list of its neighbors in feature space. RankSim is complementary to conventional imbalanced learning techniques, including re-weighting, two-stage training, and distribution smoothing, and lifts the state-of-the-art performance on three imbalanced regression benchmarks: IMDB-WIKI-DIR, AgeDB-DIR, and STS-B-DIR.

Related papers

• Contrastive Learning for Semi-Supervised Deep Regression with Generalized Ordinal Rankings from Spectral Seriation [18.192043514568187]
  We extend contrastive regression methods to allow unlabeled data to be used in the semi-supervised setting.<n>Our method can surpass existing state-of-the-art semi-supervised deep regression methods.
  arXiv  Detail & Related papers  (2025-12-10T02:45:23Z)
• Ordinality in Discrete-level Question Difficulty Estimation: Introducing Balanced DRPS and OrderedLogitNN [1.0514231683620516]
  Question difficulty is often represented using discrete levels, framing the task as ordinal regression.<n>This study addresses these limitations by benchmarking three types of model outputs.<n>We propose OrderedLogitNN, extending the ordered logit model from econometrics to neural networks.
  arXiv  Detail & Related papers  (2025-07-01T13:38:33Z)
• Sample Selection via Contrastive Fragmentation for Noisy Label Regression [38.26477510619512]
  We propose a novel approach named ConFrag, where we collectively model the regression data by transforming them into disjoint yet contrasting fragmentation pairs. Our ConFrag framework leverages a mixture of neighboring fragments to discern noisy labels through neighborhood agreement among expert feature extractors. We extensively perform experiments on six newly curated benchmark datasets of diverse domains, including age prediction, price prediction, and music production year estimation.
  arXiv  Detail & Related papers  (2025-02-25T02:04:14Z)
• Error Distribution Smoothing:Advancing Low-Dimensional Imbalanced Regression [2.435853975142516]
  In real-world regression tasks, datasets frequently exhibit imbalanced distributions, characterized by a scarcity of data in high-complexity regions and an abundance in low-complexity areas. We introduce a novel concept of Imbalanced Regression, which takes into account both the complexity of the problem and the density of data points, extending beyond traditional definitions that focus only on data density. We propose Error Distribution Smoothing (EDS) as a solution to tackle imbalanced regression, effectively selecting a representative subset from the dataset to reduce redundancy while maintaining balance and representativeness.
  arXiv  Detail & Related papers  (2025-02-04T12:40:07Z)
• Sebra: Debiasing Through Self-Guided Bias Ranking [54.09529903433859]
  Ranking samples by fine-grained estimates of spuriosity has recently been shown to significantly benefit bias mitigation. We propose a debiasing framework based on our novel ulSelf-Guided ulBias ulRanking (emphSebra) Sebra mitigates biases via an automatic ranking of data points by spuriosity within their respective classes.
  arXiv  Detail & Related papers  (2025-01-30T11:31:38Z)
• SimPro: A Simple Probabilistic Framework Towards Realistic Long-Tailed Semi-Supervised Learning [49.94607673097326]
  We propose a highly adaptable framework, designated as SimPro, which does not rely on any predefined assumptions about the distribution of unlabeled data. Our framework, grounded in a probabilistic model, innovatively refines the expectation-maximization algorithm. Our method showcases consistent state-of-the-art performance across diverse benchmarks and data distribution scenarios.
  arXiv  Detail & Related papers  (2024-02-21T03:39:04Z)
• RankMatch: A Novel Approach to Semi-Supervised Label Distribution Learning Leveraging Inter-label Correlations [52.549807652527306]
  This paper introduces RankMatch, an innovative approach for Semi-Supervised Label Distribution Learning (SSLDL) RankMatch effectively utilizes a small number of labeled examples in conjunction with a larger quantity of unlabeled data. We establish a theoretical generalization bound for RankMatch, and through extensive experiments, demonstrate its superiority in performance against existing SSLDL methods.
  arXiv  Detail & Related papers  (2023-12-11T12:47:29Z)
• Generating Unbiased Pseudo-labels via a Theoretically Guaranteed Chebyshev Constraint to Unify Semi-supervised Classification and Regression [57.17120203327993]
  threshold-to-pseudo label process (T2L) in classification uses confidence to determine the quality of label. In nature, regression also requires unbiased methods to generate high-quality labels. We propose a theoretically guaranteed constraint for generating unbiased labels based on Chebyshev's inequality.
  arXiv  Detail & Related papers  (2023-11-03T08:39:35Z)
• ConR: Contrastive Regularizer for Deep Imbalanced Regression [6.5356646210003015]
  ConR is a contrastive regularizer that models global and local label similarities in feature space. We show that ConR significantly boosts the performance of all the state-of-the-art methods on four large-scale deep imbalanced regression benchmarks.
  arXiv  Detail & Related papers  (2023-09-13T00:30:32Z)
• Neighbour Consistency Guided Pseudo-Label Refinement for Unsupervised Person Re-Identification [80.98291772215154]
  Unsupervised person re-identification (ReID) aims at learning discriminative identity features for person retrieval without any annotations. Recent advances accomplish this task by leveraging clustering-based pseudo labels. We propose a Neighbour Consistency guided Pseudo Label Refinement framework.
  arXiv  Detail & Related papers  (2022-11-30T09:39:57Z)
• Distribution-Aware Semantics-Oriented Pseudo-label for Imbalanced Semi-Supervised Learning [80.05441565830726]
  This paper addresses imbalanced semi-supervised learning, where heavily biased pseudo-labels can harm the model performance. We propose a general pseudo-labeling framework to address the bias motivated by this observation. We term the novel pseudo-labeling framework for imbalanced SSL as Distribution-Aware Semantics-Oriented (DASO) Pseudo-label.
  arXiv  Detail & Related papers  (2021-06-10T11:58:25Z)
• SimPLE: Similar Pseudo Label Exploitation for Semi-Supervised Classification [24.386165255835063]
  A common classification task situation is where one has a large amount of data available for training, but only a small portion is with class labels. The goal of semi-supervised training, in this context, is to improve classification accuracy by leverage information from a large amount of unlabeled data. We propose a novel unsupervised objective that focuses on the less studied relationship between the high confidence unlabeled data that are similar to each other. Our proposed SimPLE algorithm shows significant performance gains over previous algorithms on CIFAR-100 and Mini-ImageNet, and is on par with the state-of-the-art methods
  arXiv  Detail & Related papers  (2021-03-30T23:48:06Z)
• Imbalanced Continual Learning with Partitioning Reservoir Sampling [46.427023757892336]
  Continual learning from a sequential stream of data is a crucial challenge for machine learning research. We identify unanticipated adversity innately existent in many multi-label datasets, the long-tailed distribution. We propose a new sampling strategy for replay-based approach named Partitioning Reservoir Sampling (PRS)
  arXiv  Detail & Related papers  (2020-09-08T10:28:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.