Learning Label Hierarchy with Supervised Contrastive Learning

• URL: http://arxiv.org/abs/2402.00232v1
• Date: Wed, 31 Jan 2024 23:21:40 GMT
• Title: Learning Label Hierarchy with Supervised Contrastive Learning
• Authors: Ruixue Lian, William A. Sethares, Junjie Hu
• Abstract summary: Supervised contrastive learning (SCL) frameworks treat each class as independent and thus consider all classes to be equally important. This paper introduces a family of Label-Aware SCL methods (LASCL) that incorporates hierarchical information to SCL by leveraging similarities between classes. Experiments on three datasets show that the proposed LASCL works well on text classification of distinguishing a single label among multi-labels.
• Score: 8.488965459026678
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Supervised contrastive learning (SCL) frameworks treat each class as independent and thus consider all classes to be equally important. This neglects the common scenario in which label hierarchy exists, where fine-grained classes under the same category show more similarity than very different ones. This paper introduces a family of Label-Aware SCL methods (LASCL) that incorporates hierarchical information to SCL by leveraging similarities between classes, resulting in creating a more well-structured and discriminative feature space. This is achieved by first adjusting the distance between instances based on measures of the proximity of their classes with the scaled instance-instance-wise contrastive. An additional instance-center-wise contrastive is introduced to move within-class examples closer to their centers, which are represented by a set of learnable label parameters. The learned label parameters can be directly used as a nearest neighbor classifier without further finetuning. In this way, a better feature representation is generated with improvements of intra-cluster compactness and inter-cluster separation. Experiments on three datasets show that the proposed LASCL works well on text classification of distinguishing a single label among multi-labels, outperforming the baseline supervised approaches. Our code is publicly available.

Related papers

• Label Cluster Chains for Multi-Label Classification [2.072831155509228]
  Multi-label classification is a type of supervised machine learning that can simultaneously assign multiple labels to an instance. We propose a method to chain disjoint correlated label clusters obtained by applying a partition method in the label space. Our proposal shows that learning and chaining disjoint correlated label clusters can better explore and learn label correlations.
  arXiv  Detail & Related papers  (2024-11-01T11:16:37Z)
• Logit Separability-Driven Samples and Multiple Class-Related Words Selection for Advancing In-Context Learning [0.0]
  We introduce logit separability, a criterion to assess the clarity of both samples and class-related words at the logit level. We find that incorporating multiple class-related words for each sample, rather than relying on a single class name, improves performance by offering a broader range of label information. We propose LICL, a logit separability-based method that jointly organizes samples and integrates multiple class-related words into each sample-label pair.
  arXiv  Detail & Related papers  (2024-06-16T12:11:46Z)
• TagCLIP: A Local-to-Global Framework to Enhance Open-Vocabulary Multi-Label Classification of CLIP Without Training [29.431698321195814]
  Contrastive Language-Image Pre-training (CLIP) has demonstrated impressive capabilities in open-vocabulary classification. CLIP shows poor performance on multi-label datasets because the global feature tends to be dominated by the most prominent class. We propose a local-to-global framework to obtain image tags.
  arXiv  Detail & Related papers  (2023-12-20T08:15:40Z)
• Generalized Category Discovery with Clustering Assignment Consistency [56.92546133591019]
  Generalized category discovery (GCD) is a recently proposed open-world task. We propose a co-training-based framework that encourages clustering consistency. Our method achieves state-of-the-art performance on three generic benchmarks and three fine-grained visual recognition datasets.
  arXiv  Detail & Related papers  (2023-10-30T00:32:47Z)
• Triplet Contrastive Learning for Unsupervised Vehicle Re-identification [55.445358749042384]
  Part feature learning is a critical technology for fine semantic understanding in vehicle re-identification. We propose a novel Triplet Contrastive Learning framework (TCL) which leverages cluster features to bridge the part features and global features.
  arXiv  Detail & Related papers  (2023-01-23T15:52:12Z)
• Learning Disentangled Label Representations for Multi-label Classification [39.97251974500034]
  One-shared-Feature-for-Multiple-Labels (OFML) is not conducive to learning discriminative label features. We introduce the One-specific-Feature-for-One-Label (OFOL) mechanism and propose a novel disentangled label feature learning framework. We achieve state-of-the-art performance on eight datasets.
  arXiv  Detail & Related papers  (2022-12-02T21:49:34Z)
• Multiple Instance Learning via Iterative Self-Paced Supervised Contrastive Learning [22.07044031105496]
  Learning representations for individual instances when only bag-level labels are available is a challenge in multiple instance learning (MIL) We propose a novel framework, Iterative Self-paced Supervised Contrastive Learning for MIL Representations (ItS2CLR) It improves the learned representation by exploiting instance-level pseudo labels derived from the bag-level labels.
  arXiv  Detail & Related papers  (2022-10-17T21:43:32Z)
• Learning to Detect Instance-level Salient Objects Using Complementary Image Labels [55.049347205603304]
  We present the first weakly-supervised approach to the salient instance detection problem. We propose a novel weakly-supervised network with three branches: a Saliency Detection Branch leveraging class consistency information to locate candidate objects; a Boundary Detection Branch exploiting class discrepancy information to delineate object boundaries; and a Centroid Detection Branch using subitizing information to detect salient instance centroids.
  arXiv  Detail & Related papers  (2021-11-19T10:15:22Z)
• Neighborhood Contrastive Learning for Novel Class Discovery [79.14767688903028]
  We build a new framework, named Neighborhood Contrastive Learning, to learn discriminative representations that are important to clustering performance. We experimentally demonstrate that these two ingredients significantly contribute to clustering performance and lead our model to outperform state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2021-06-20T17:34:55Z)
• Generative Multi-Label Zero-Shot Learning [136.17594611722285]
  Multi-label zero-shot learning strives to classify images into multiple unseen categories for which no data is available during training. Our work is the first to tackle the problem of multi-label feature in the (generalized) zero-shot setting. Our cross-level fusion-based generative approach outperforms the state-of-the-art on all three datasets.
  arXiv  Detail & Related papers  (2021-01-27T18:56:46Z)
• Unsupervised Feature Learning by Cross-Level Instance-Group Discrimination [68.83098015578874]
  We integrate between-instance similarity into contrastive learning, not directly by instance grouping, but by cross-level discrimination. CLD effectively brings unsupervised learning closer to natural data and real-world applications. New state-of-the-art on self-supervision, semi-supervision, and transfer learning benchmarks, and beats MoCo v2 and SimCLR on every reported performance.
  arXiv  Detail & Related papers  (2020-08-09T21:13:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.