Leveraging Group Classification with Descending Soft Labeling for Deep Imbalanced Regression

• URL: http://arxiv.org/abs/2412.12327v2
• Date: Thu, 19 Dec 2024 09:34:30 GMT
• Title: Leveraging Group Classification with Descending Soft Labeling for Deep Imbalanced Regression
• Authors: Ruizhi Pu, Gezheng Xu, Ruiyi Fang, Binkun Bao, Charles X. Ling, Boyu Wang,
• Abstract summary: Deep imbalanced regression (DIR) is an intriguing yet under-explored problem in machine learning. We first bridge the connection between the objectives of DIR and classification from a Bayesian perspective. Specifically, by aggregating the data at nearby labels into the same groups, we introduce an ordinal group-aware contrastive learning loss. We also propose a symmetric descending soft labeling strategy to exploit the intrinsic similarity across the data.
• Score: 6.649953811669191
• License:
• Abstract: Deep imbalanced regression (DIR), where the target values have a highly skewed distribution and are also continuous, is an intriguing yet under-explored problem in machine learning. While recent works have already shown that incorporating various classification-based regularizers can produce enhanced outcomes, the role of classification remains elusive in DIR. Moreover, such regularizers (e.g., contrastive penalties) merely focus on learning discriminative features of data, which inevitably results in ignorance of either continuity or similarity across the data. To address these issues, we first bridge the connection between the objectives of DIR and classification from a Bayesian perspective. Consequently, this motivates us to decompose the objective of DIR into a combination of classification and regression tasks, which naturally guides us toward a divide-and-conquer manner to solve the DIR problem. Specifically, by aggregating the data at nearby labels into the same groups, we introduce an ordinal group-aware contrastive learning loss along with a multi-experts regressor to tackle the different groups of data thereby maintaining the data continuity. Meanwhile, considering the similarity between the groups, we also propose a symmetric descending soft labeling strategy to exploit the intrinsic similarity across the data, which allows classification to facilitate regression more effectively. Extensive experiments on real-world datasets also validate the effectiveness of our method.

Related papers

• Multi-Label Bayesian Active Learning with Inter-Label Relationships [3.88369051454137]
  We propose a new multi-label active learning strategy to address both challenges. Our method incorporates progressively updated positive and negative correlation matrices to capture co-occurrence and disjoint relationships. Our strategy consistently achieves more reliable and superior performance, compared to several established methods.
  arXiv  Detail & Related papers  (2024-11-26T23:28:54Z)
• Decoupling the Class Label and the Target Concept in Machine Unlearning [81.69857244976123]
  Machine unlearning aims to adjust a trained model to approximate a retrained one that excludes a portion of training data. Previous studies showed that class-wise unlearning is successful in forgetting the knowledge of a target class. We propose a general framework, namely, TARget-aware Forgetting (TARF)
  arXiv  Detail & Related papers  (2024-06-12T14:53:30Z)
• Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation [63.180725016463974]
  Cross-modal retrieval relies on well-matched large-scale datasets that are laborious in practice. We introduce a novel noisy correspondence learning framework, namely textbfSelf-textbfReinforcing textbfErrors textbfMitigation (SREM)
  arXiv  Detail & Related papers  (2023-12-27T09:03:43Z)
• Tackling Diverse Minorities in Imbalanced Classification [80.78227787608714]
  Imbalanced datasets are commonly observed in various real-world applications, presenting significant challenges in training classifiers. We propose generating synthetic samples iteratively by mixing data samples from both minority and majority classes. We demonstrate the effectiveness of our proposed framework through extensive experiments conducted on seven publicly available benchmark datasets.
  arXiv  Detail & Related papers  (2023-08-28T18:48:34Z)
• Unilaterally Aggregated Contrastive Learning with Hierarchical Augmentation for Anomaly Detection [64.50126371767476]
  We propose Unilaterally Aggregated Contrastive Learning with Hierarchical Augmentation (UniCon-HA) We explicitly encourage the concentration of inliers and the dispersion of virtual outliers via supervised and unsupervised contrastive losses. Our method is evaluated under three AD settings including unlabeled one-class, unlabeled multi-class, and labeled multi-class.
  arXiv  Detail & Related papers  (2023-08-20T04:01:50Z)
• CADIS: Handling Cluster-skewed Non-IID Data in Federated Learning with Clustered Aggregation and Knowledge DIStilled Regularization [3.3711670942444014]
  Federated learning enables edge devices to train a global model collaboratively without exposing their data. We tackle a new type of Non-IID data, called cluster-skewed non-IID, discovered in actual data sets. We propose an aggregation scheme that guarantees equality between clusters.
  arXiv  Detail & Related papers  (2023-02-21T02:53:37Z)
• Deep Clustering based Fair Outlier Detection [19.601280507914325]
  We propose an instance-level weighted representation learning strategy to enhance the joint deep clustering and outlier detection. Our DCFOD method consistently achieves superior performance on both the outlier detection validity and two types of fairness notions in outlier detection.
  arXiv  Detail & Related papers  (2021-06-09T15:12:26Z)
• Learning from Self-Discrepancy via Multiple Co-teaching for Cross-Domain Person Re-Identification [12.106894735305714]
  We propose a multiple co-teaching framework for domain adaptive person re-ID. Our method achieves competitive performance compared with the state-of-the-arts.
  arXiv  Detail & Related papers  (2021-04-06T03:12:11Z)
• Delving into Deep Imbalanced Regression [41.90687097747504]
  We define Deep Imbalanced Regression (DIR) as learning from imbalanced data with continuous targets. Motivated by the intrinsic difference between categorical and continuous label space, we propose distribution smoothing for both labels and features. Our work fills the gap in benchmarks and techniques for practical imbalanced regression problems.
  arXiv  Detail & Related papers  (2021-02-18T18:56:03Z)
• ORDisCo: Effective and Efficient Usage of Incremental Unlabeled Data for Semi-supervised Continual Learning [52.831894583501395]
  Continual learning assumes the incoming data are fully labeled, which might not be applicable in real applications. We propose deep Online Replay with Discriminator Consistency (ORDisCo) to interdependently learn a classifier with a conditional generative adversarial network (GAN) We show ORDisCo achieves significant performance improvement on various semi-supervised learning benchmark datasets for SSCL.
  arXiv  Detail & Related papers  (2021-01-02T09:04:14Z)
• Adversarial Self-Supervised Contrastive Learning [62.17538130778111]
  Existing adversarial learning approaches mostly use class labels to generate adversarial samples that lead to incorrect predictions. We propose a novel adversarial attack for unlabeled data, which makes the model confuse the instance-level identities of the perturbed data samples. We present a self-supervised contrastive learning framework to adversarially train a robust neural network without labeled data.
  arXiv  Detail & Related papers  (2020-06-13T08:24:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.