Solving the long-tailed distribution problem by exploiting the synergies and balance of different techniques

• URL: http://arxiv.org/abs/2501.13756v1
• Date: Thu, 23 Jan 2025 15:35:15 GMT
• Title: Solving the long-tailed distribution problem by exploiting the synergies and balance of different techniques
• Authors: Ziheng Wang, Toni Lassila, Sharib Ali,
• Abstract summary: Long-tailed data distribution is common, making it challenging for models trained on empirical risk minimisation to learn and classify tail classes effectively. Our study delves into three long-tail recognition techniques: Supervised Contrastive Learning (SCL), Rare-Class Sample Generator (RSG), and Label-Distribution-Aware Margin Loss (LDAM)
• Score: 6.068761166911576
• License:
• Abstract: In real-world data, long-tailed data distribution is common, making it challenging for models trained on empirical risk minimisation to learn and classify tail classes effectively. While many studies have sought to improve long tail recognition by altering the data distribution in the feature space and adjusting model decision boundaries, research on the synergy and corrective approach among various methods is limited. Our study delves into three long-tail recognition techniques: Supervised Contrastive Learning (SCL), Rare-Class Sample Generator (RSG), and Label-Distribution-Aware Margin Loss (LDAM). SCL enhances intra-class clusters based on feature similarity and promotes clear inter-class separability but tends to favour dominant classes only. When RSG is integrated into the model, we observed that the intra-class features further cluster towards the class centre, which demonstrates a synergistic effect together with SCL's principle of enhancing intra-class clustering. RSG generates new tail features and compensates for the tail feature space squeezed by SCL. Similarly, LDAM is known to introduce a larger margin specifically for tail classes; we demonstrate that LDAM further bolsters the model's performance on tail classes when combined with the more explicit decision boundaries achieved by SCL and RSG. Furthermore, SCL can compensate for the dominant class accuracy sacrificed by RSG and LDAM. Our research emphasises the synergy and balance among the three techniques, with each amplifying the strengths of the others and mitigating their shortcomings. Our experiment on long-tailed distribution datasets, using an end-to-end architecture, yields competitive results by enhancing tail class accuracy without compromising dominant class performance, achieving a balanced improvement across all classes.

Related papers

• Long-Tail Learning with Rebalanced Contrastive Loss [1.4443576276330394]
  We present Rebalanced Contrastive Learning (RCL), an efficient means to increase the long tail classification accuracy. RCL addresses three main aspects: Feature space balancedness, Intra-Class compactness and Regularization. Our experiments on three benchmark datasets demonstrate the richness of the learnt embeddings and increased top-1 balanced accuracy RCL provides to the BCL framework.
  arXiv  Detail & Related papers  (2023-12-04T09:27:03Z)
• Uncertainty-guided Boundary Learning for Imbalanced Social Event Detection [64.4350027428928]
  We propose a novel uncertainty-guided class imbalance learning framework for imbalanced social event detection tasks. Our model significantly improves social event representation and classification tasks in almost all classes, especially those uncertain ones.
  arXiv  Detail & Related papers  (2023-10-30T03:32:04Z)
• Combating Representation Learning Disparity with Geometric Harmonization [50.29859682439571]
  We propose a novel Geometric Harmonization (GH) method to encourage category-level uniformity in representation learning. Our proposal does not alter the setting of SSL and can be easily integrated into existing methods in a low-cost manner.
  arXiv  Detail & Related papers  (2023-10-26T17:41:11Z)
• A dual-branch model with inter- and intra-branch contrastive loss for long-tailed recognition [7.225494453600985]
  Models trained on long-tailed datasets have poor adaptability to tail classes and the decision boundaries are ambiguous. We propose a simple yet effective model, named Dual-Branch Long-Tailed Recognition (DB-LTR), which includes an imbalanced learning branch and a Contrastive Learning Branch (CoLB) CoLB can improve the capability of the model in adapting to tail classes and assist the imbalanced learning branch to learn a well-represented feature space and discriminative decision boundary.
  arXiv  Detail & Related papers  (2023-09-28T03:31:11Z)
• Dual Compensation Residual Networks for Class Imbalanced Learning [98.35401757647749]
  We propose Dual Compensation Residual Networks to better fit both tail and head classes. An important factor causing overfitting is that there is severe feature drift between training and test data on tail classes. We also propose a Residual Balanced Multi-Proxies classifier to alleviate the under-fitting issue.
  arXiv  Detail & Related papers  (2023-08-25T04:06:30Z)
• ECL: Class-Enhancement Contrastive Learning for Long-tailed Skin Lesion Classification [7.7379419801373475]
  Skin image datasets often suffer from imbalanced data distribution, exacerbating the difficulty of computer-aided skin disease diagnosis. We propose class-Enhancement Contrastive Learning (ECL), which enriches the information of minority classes and treats different classes equally.
  arXiv  Detail & Related papers  (2023-07-09T09:29:15Z)
• Meta-Causal Feature Learning for Out-of-Distribution Generalization [71.38239243414091]
  This paper presents a balanced meta-causal learner (BMCL), which includes a balanced task generation module (BTG) and a meta-causal feature learning module (MCFL) BMCL effectively identifies the class-invariant visual regions for classification and may serve as a general framework to improve the performance of the state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-08-22T09:07:02Z)
• Improving GANs for Long-Tailed Data through Group Spectral Regularization [51.58250647277375]
  We propose a novel group Spectral Regularizer (gSR) that prevents the spectral explosion alleviating mode collapse. We find that gSR effectively combines with existing augmentation and regularization techniques, leading to state-of-the-art image generation performance on long-tailed data.
  arXiv  Detail & Related papers  (2022-08-21T17:51:05Z)
• Long-Tailed Classification with Gradual Balanced Loss and Adaptive Feature Generation [19.17617301462919]
  We propose a new method, Gradual Balanced Loss and Adaptive Feature Generator (GLAG) to alleviate imbalance. State-of-the-art results have been achieved on long-tail datasets such as CIFAR100-LT, ImageNetLT, and iNaturalist.
  arXiv  Detail & Related papers  (2022-02-28T01:20:35Z)
• Targeted Supervised Contrastive Learning for Long-Tailed Recognition [50.24044608432207]
  Real-world data often exhibits long tail distributions with heavy class imbalance. We show that while supervised contrastive learning can help improve performance, past baselines suffer from poor uniformity brought in by imbalanced data distribution. We propose targeted supervised contrastive learning (TSC), which improves the uniformity of the feature distribution on the hypersphere.
  arXiv  Detail & Related papers  (2021-11-27T22:40:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.