ConR: Contrastive Regularizer for Deep Imbalanced Regression

• URL: http://arxiv.org/abs/2309.06651v4
• Date: Wed, 13 Mar 2024 19:10:48 GMT
• Title: ConR: Contrastive Regularizer for Deep Imbalanced Regression
• Abstract summary: ConR is a contrastive regularizer that models global and local label similarities in feature space. We show that ConR significantly boosts the performance of all the state-of-the-art methods on four large-scale deep imbalanced regression benchmarks.
• Score: 6.5356646210003015
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Imbalanced distributions are ubiquitous in real-world data. They create constraints on Deep Neural Networks to represent the minority labels and avoid bias towards majority labels. The extensive body of imbalanced approaches address categorical label spaces but fail to effectively extend to regression problems where the label space is continuous. Local and global correlations among continuous labels provide valuable insights towards effectively modelling relationships in feature space. In this work, we propose ConR, a contrastive regularizer that models global and local label similarities in feature space and prevents the features of minority samples from being collapsed into their majority neighbours. ConR discerns the disagreements between the label space and feature space and imposes a penalty on these disagreements. ConR addresses the continuous nature of label space with two main strategies in a contrastive manner: incorrect proximities are penalized proportionate to the label similarities and the correct ones are encouraged to model local similarities. ConR consolidates essential considerations into a generic, easy-to-integrate, and efficient method that effectively addresses deep imbalanced regression. Moreover, ConR is orthogonal to existing approaches and smoothly extends to uni- and multi-dimensional label spaces. Our comprehensive experiments show that ConR significantly boosts the performance of all the state-of-the-art methods on four large-scale deep imbalanced regression benchmarks. Our code is publicly available in https://github.com/BorealisAI/ConR.

Related papers

• Unbiased Open World Regularization for Fair Self-Supervised Learning [11.74837809839014]
  Self-supervised learning models are susceptible to learning spurious biases in the dataset.<n>We propose Unbiased Open World Regularization (UOWReg), an encoder-only framework.<n>We empirically validate this framework across both Gaussian and spherical latent spaces.
  arXiv  Detail & Related papers  (2026-07-24T09:51:22Z)
• Shortcut to Nowhere: Demystifying Deep Spurious Regression [4.283376582002132]
  We define Deep Spurious Regression (DSR) as learning from regression data with attribute-label confounding.<n>Motivated by the intrinsic difference between classification and regression shortcuts, we propose to exploit the similarity among spurious attributes in both label and feature spaces.<n>Our work fills the gap in benchmarks and techniques for studying spurious correlations in continuous prediction.
  arXiv  Detail & Related papers  (2026-06-01T05:40:11Z)
• Beyond Distribution Estimation: Simplex Anchored Structural Inference Towards Universal Semi-Supervised Learning [47.92504362652887]
  Semi-supervised learning faces significant challenges in realistic scenarios where labeled data is scarce and unlabeled data follows unknown, arbitrary distributions.<n>We formalize this critical yet under-explored paradigm as Universal Semi-supervised Learning (UniSSL)<n>We introduce a weighting strategy based on distribution-agnostic metrics to prioritize reliable pseudo-labels and an auxiliary branch to isolate potentially erroneous pseudo-labels.
  arXiv  Detail & Related papers  (2026-05-08T10:33:09Z)
• Semi-Supervised Regression with Heteroscedastic Pseudo-Labels [50.54050677867914]
  We propose an uncertainty-aware pseudo-labeling framework that dynamically adjusts pseudo-label influence from a bi-level optimization perspective.<n>We provide theoretical insights and extensive experiments to validate our approach across various benchmark SSR datasets.
  arXiv  Detail & Related papers  (2025-10-17T03:06:23Z)
• Reliable Active Learning from Unreliable Labels via Neural Collapse Geometry [5.1511135538176]
  Active Learning (AL) promises to reduce annotation cost by prioritizing informative samples, yet its reliability is undermined when labels are noisy or when the data distribution shifts.<n>We propose Active Learning via Neural Collapse Geometry (NCAL-R), a framework that leverages the emergent geometric regularities of deep networks to counteract unreliable supervision.
  arXiv  Detail & Related papers  (2025-10-10T17:50:31Z)
• Online hierarchical partitioning of the output space in extreme multi-label data stream [2.474908349649168]
  This work introduces iHOMER, an online multi-label learning framework that partitions the label space into disjoint clusters, correlated without relying on predefined hierarchies.<n>Experiments across 23 real-world datasets show iHOMER outperforms 5 state-of-the-art global baselines, such as MLHAT, MLHT of Pruned Sets and iSOUPT, by 23%, and 12 local baselines, such as binary relevance transformations of kNN, EFDT, ARF, and ADWIN bagging/boosting ensembles, by 32%, establishing its robustness for online multi-label classification.
  arXiv  Detail & Related papers  (2025-07-28T14:47:13Z)
• Towards Self-Supervised Covariance Estimation in Deep Heteroscedastic Regression [102.24287051757469]
  We study self-supervised covariance estimation in deep heteroscedastic regression. We derive an upper bound on the 2-Wasserstein distance between normal distributions. Experiments over a wide range of synthetic and real datasets demonstrate that the proposed 2-Wasserstein bound coupled with pseudo label annotations results in a computationally cheaper yet accurate deep heteroscedastic regression.
  arXiv  Detail & Related papers  (2025-02-14T22:37:11Z)
• Error Distribution Smoothing:Advancing Low-Dimensional Imbalanced Regression [2.435853975142516]
  In real-world regression tasks, datasets frequently exhibit imbalanced distributions, characterized by a scarcity of data in high-complexity regions and an abundance in low-complexity areas. We introduce a novel concept of Imbalanced Regression, which takes into account both the complexity of the problem and the density of data points, extending beyond traditional definitions that focus only on data density. We propose Error Distribution Smoothing (EDS) as a solution to tackle imbalanced regression, effectively selecting a representative subset from the dataset to reduce redundancy while maintaining balance and representativeness.
  arXiv  Detail & Related papers  (2025-02-04T12:40:07Z)
• Exploring Homogeneous and Heterogeneous Consistent Label Associations for Unsupervised Visible-Infrared Person ReID [57.500045584556794]
  We introduce a Modality-Unified Label Transfer (MULT) module that simultaneously accounts for both homogeneous and heterogeneous fine-grained instance-level structures. The proposed MULT ensures that the generated pseudo-labels maintain alignment across modalities while upholding structural consistency within intra-modality. Experiments demonstrate that our proposed method outperforms existing state-of-the-art USL-VI-ReID methods.
  arXiv  Detail & Related papers  (2024-02-01T15:33:17Z)
• RankMatch: A Novel Approach to Semi-Supervised Label Distribution Learning Leveraging Inter-label Correlations [52.549807652527306]
  This paper introduces RankMatch, an innovative approach for Semi-Supervised Label Distribution Learning (SSLDL) RankMatch effectively utilizes a small number of labeled examples in conjunction with a larger quantity of unlabeled data. We establish a theoretical generalization bound for RankMatch, and through extensive experiments, demonstrate its superiority in performance against existing SSLDL methods.
  arXiv  Detail & Related papers  (2023-12-11T12:47:29Z)
• Generating Unbiased Pseudo-labels via a Theoretically Guaranteed Chebyshev Constraint to Unify Semi-supervised Classification and Regression [57.17120203327993]
  threshold-to-pseudo label process (T2L) in classification uses confidence to determine the quality of label. In nature, regression also requires unbiased methods to generate high-quality labels. We propose a theoretically guaranteed constraint for generating unbiased labels based on Chebyshev's inequality.
  arXiv  Detail & Related papers  (2023-11-03T08:39:35Z)
• All Points Matter: Entropy-Regularized Distribution Alignment for Weakly-supervised 3D Segmentation [67.30502812804271]
  Pseudo-labels are widely employed in weakly supervised 3D segmentation tasks where only sparse ground-truth labels are available for learning. We propose a novel learning strategy to regularize the generated pseudo-labels and effectively narrow the gaps between pseudo-labels and model predictions.
  arXiv  Detail & Related papers  (2023-05-25T08:19:31Z)
• RankSim: Ranking Similarity Regularization for Deep Imbalanced Regression [48.58405556678924]
  RankSim encodes an inductive bias that samples closer in label space should also be closer in feature space. RankSim is complementary to conventional imbalanced learning techniques.
  arXiv  Detail & Related papers  (2022-05-30T16:51:25Z)
• Part-based Pseudo Label Refinement for Unsupervised Person Re-identification [29.034390810078172]
  Unsupervised person re-identification (re-ID) aims at learning discriminative representations for person retrieval from unlabeled data. Recent techniques accomplish this task by using pseudo-labels, but these labels are inherently noisy and deteriorate the accuracy. We propose a novel Part-based Pseudo Label Refinement (PPLR) framework that reduces the label noise by employing the complementary relationship between global and part features.
  arXiv  Detail & Related papers  (2022-03-28T12:15:53Z)
• Distribution-Aware Semantics-Oriented Pseudo-label for Imbalanced Semi-Supervised Learning [80.05441565830726]
  This paper addresses imbalanced semi-supervised learning, where heavily biased pseudo-labels can harm the model performance. We propose a general pseudo-labeling framework to address the bias motivated by this observation. We term the novel pseudo-labeling framework for imbalanced SSL as Distribution-Aware Semantics-Oriented (DASO) Pseudo-label.
  arXiv  Detail & Related papers  (2021-06-10T11:58:25Z)
• PLM: Partial Label Masking for Imbalanced Multi-label Classification [59.68444804243782]
  Neural networks trained on real-world datasets with long-tailed label distributions are biased towards frequent classes and perform poorly on infrequent classes. We propose a method, Partial Label Masking (PLM), which utilizes this ratio during training. Our method achieves strong performance when compared to existing methods on both multi-label (MultiMNIST and MSCOCO) and single-label (imbalanced CIFAR-10 and CIFAR-100) image classification datasets.
  arXiv  Detail & Related papers  (2021-05-22T18:07:56Z)
• Delving into Deep Imbalanced Regression [41.90687097747504]
  We define Deep Imbalanced Regression (DIR) as learning from imbalanced data with continuous targets. Motivated by the intrinsic difference between categorical and continuous label space, we propose distribution smoothing for both labels and features. Our work fills the gap in benchmarks and techniques for practical imbalanced regression problems.
  arXiv  Detail & Related papers  (2021-02-18T18:56:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.