Similarity-Dissimilarity Loss for Multi-label Supervised Contrastive Learning

• URL: http://arxiv.org/abs/2410.13439v3
• Date: Tue, 25 Mar 2025 21:47:03 GMT
• Title: Similarity-Dissimilarity Loss for Multi-label Supervised Contrastive Learning
• Authors: Guangming Huang, Yunfei Long, Cunjin Luo,
• Abstract summary: Supervised contrastive learning has achieved remarkable success by leveraging label information.<n>However, determining positive samples in multi-label scenarios remains a critical challenge.
• Score: 4.325075044327162
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Supervised contrastive learning has achieved remarkable success by leveraging label information; however, determining positive samples in multi-label scenarios remains a critical challenge. In multi-label supervised contrastive learning (MSCL), relations among multi-label samples are not yet fully defined, leading to ambiguity in identifying positive samples and formulating contrastive loss functions to construct the representation space. To address these challenges, we: (i) first define five distinct multi-label relations in MSCL to systematically identify positive samples, (ii) introduce a novel Similarity-Dissimilarity Loss that dynamically re-weights samples through computing the similarity and dissimilarity factors between positive samples and given anchors based on multi-label relations, and (iii) further provide theoretical grounded proof for our method through rigorous mathematical analysis that supports the formulation and effectiveness of the proposed loss function. We conduct the experiments across both image and text modalities, and extend the evaluation to medical domain. The results demonstrate that our method consistently outperforms baselines in a comprehensive evaluation, confirming its effectiveness and robustness. Code is available at: https://github.com/guangminghuang/similarity-dissimilarity-loss.

Related papers

• Style-Aware Blending and Prototype-Based Cross-Contrast Consistency for Semi-Supervised Medical Image Segmentation [5.430389901321259]
  We propose a style-aware blending and prototype-based cross-contrast consistency learning framework.<n>Inspired by the empirical observation that the distribution mismatch between labeled and unlabeled data can be characterized by statistical moments, we design a style-guided distribution blending module.<n>Considering the potential noise in strong pseudo-labels, we introduce a prototype-based cross-contrast strategy to encourage the model to learn informative supervisory signals.
  arXiv  Detail & Related papers  (2025-07-28T11:26:24Z)
• Confidence-Aware Self-Distillation for Multimodal Sentiment Analysis with Incomplete Modalities [15.205192581534973]
  Multimodal sentiment analysis aims to understand human sentiment through multimodal data.<n>Existing methods for handling modality missingness are based on data reconstruction or common subspace projections.<n>We propose a Confidence-Aware Self-Distillation (CASD) strategy that effectively incorporates multimodal probabilistic embeddings.
  arXiv  Detail & Related papers  (2025-06-02T09:48:41Z)
• Mutual Evidential Deep Learning for Medical Image Segmentation [39.930548790471896]
  We propose a mutual evidential deep learning framework that offers a potentially viable solution for pseudo-label generation in semi-supervised learning.<n>We show that MEDL achieves state-of-the-art performance in experiments on five mainstream datasets.
  arXiv  Detail & Related papers  (2025-05-18T13:42:27Z)
• Multi-Label Bayesian Active Learning with Inter-Label Relationships [3.88369051454137]
  We propose a new multi-label active learning strategy to address both challenges. Our method incorporates progressively updated positive and negative correlation matrices to capture co-occurrence and disjoint relationships. Our strategy consistently achieves more reliable and superior performance, compared to several established methods.
  arXiv  Detail & Related papers  (2024-11-26T23:28:54Z)
• Dual-Decoupling Learning and Metric-Adaptive Thresholding for Semi-Supervised Multi-Label Learning [81.83013974171364]
  Semi-supervised multi-label learning (SSMLL) is a powerful framework for leveraging unlabeled data to reduce the expensive cost of collecting precise multi-label annotations. Unlike semi-supervised learning, one cannot select the most probable label as the pseudo-label in SSMLL due to multiple semantics contained in an instance. We propose a dual-perspective method to generate high-quality pseudo-labels.
  arXiv  Detail & Related papers  (2024-07-26T09:33:53Z)
• Correlation-Decoupled Knowledge Distillation for Multimodal Sentiment Analysis with Incomplete Modalities [16.69453837626083]
  We propose a Correlation-decoupled Knowledge Distillation (CorrKD) framework for the Multimodal Sentiment Analysis (MSA) task under uncertain missing modalities. We present a sample-level contrastive distillation mechanism that transfers comprehensive knowledge containing cross-sample correlations to reconstruct missing semantics. We design a response-disentangled consistency distillation strategy to optimize the sentiment decision boundaries of the student network.
  arXiv  Detail & Related papers  (2024-04-25T09:35:09Z)
• Exploring Contrastive Learning for Long-Tailed Multi-Label Text Classification [48.81069245141415]
  We introduce a novel contrastive loss function for multi-label text classification. It attains Micro-F1 scores that either match or surpass those obtained with other frequently employed loss functions. It demonstrates a significant improvement in Macro-F1 scores across three multi-label datasets.
  arXiv  Detail & Related papers  (2024-04-12T11:12:16Z)
• ProbMCL: Simple Probabilistic Contrastive Learning for Multi-label Visual Classification [16.415582577355536]
  Multi-label image classification presents a challenging task in many domains, including computer vision and medical imaging. Recent advancements have introduced graph-based and transformer-based methods to improve performance and capture label dependencies. We propose Probabilistic Multi-label Contrastive Learning (ProbMCL), a novel framework to address these challenges.
  arXiv  Detail & Related papers  (2024-01-02T22:15:20Z)
• CLAF: Contrastive Learning with Augmented Features for Imbalanced Semi-Supervised Learning [40.5117833362268]
  Semi-supervised learning and contrastive learning have been progressively combined to achieve better performances in popular applications. One common manner is assigning pseudo-labels to unlabeled samples and selecting positive and negative samples from pseudo-labeled samples to apply contrastive learning. We propose Contrastive Learning with Augmented Features (CLAF) to alleviate the scarcity of minority class samples in contrastive learning.
  arXiv  Detail & Related papers  (2023-12-15T08:27:52Z)
• Cross-Attention is Not Enough: Incongruity-Aware Dynamic Hierarchical Fusion for Multimodal Affect Recognition [69.32305810128994]
  Incongruity between modalities poses a challenge for multimodal fusion, especially in affect recognition. We propose the Hierarchical Crossmodal Transformer with Dynamic Modality Gating (HCT-DMG), a lightweight incongruity-aware model. HCT-DMG: 1) outperforms previous multimodal models with a reduced size of approximately 0.8M parameters; 2) recognizes hard samples where incongruity makes affect recognition difficult; 3) mitigates the incongruity at the latent level in crossmodal attention.
  arXiv  Detail & Related papers  (2023-05-23T01:24:15Z)
• Class-Distribution-Aware Pseudo Labeling for Semi-Supervised Multi-Label Learning [97.88458953075205]
  Pseudo-labeling has emerged as a popular and effective approach for utilizing unlabeled data. This paper proposes a novel solution called Class-Aware Pseudo-Labeling (CAP) that performs pseudo-labeling in a class-aware manner.
  arXiv  Detail & Related papers  (2023-05-04T12:52:18Z)
• Towards Integration of Discriminability and Robustness for Document-Level Relation Extraction [41.51148745387936]
  Document-level relation extraction (DocRE) predicts relations for entity pairs that rely on long-range context-dependent reasoning in a document. In this work, we aim to achieve better integration of both the discriminability and robustness for the DocRE problem. We innovatively customize entropy minimization and supervised contrastive learning for the challenging multi-label and long-tailed learning problems.
  arXiv  Detail & Related papers  (2023-04-03T09:11:18Z)
• Cluster-aware Contrastive Learning for Unsupervised Out-of-distribution Detection [0.0]
  Unsupervised out-of-distribution (OOD) Detection aims to separate the samples falling outside the distribution of training data without label information. We propose Cluster-aware Contrastive Learning (CCL) framework for unsupervised OOD detection, which considers both instance-level and semantic-level information.
  arXiv  Detail & Related papers  (2023-02-06T07:21:03Z)
• Uncertain Facial Expression Recognition via Multi-task Assisted Correction [43.02119884581332]
  We propose a novel method of multi-task assisted correction in addressing uncertain facial expression recognition called MTAC. Specifically, a confidence estimation block and a weighted regularization module are applied to highlight solid samples and suppress uncertain samples in every batch. Experiments on RAF-DB, AffectNet, and AffWild2 datasets demonstrate that the MTAC obtains substantial improvements over baselines when facing synthetic and real uncertainties.
  arXiv  Detail & Related papers  (2022-12-14T10:28:08Z)
• An Effective Deployment of Contrastive Learning in Multi-label Text Classification [6.697876965452054]
  We propose five novel contrastive losses for multi-label text classification tasks. These are Strict Contrastive Loss (SCL), Intra-label Contrastive Loss (ICL), Jaccard Similarity Contrastive Loss (JSCL), Jaccard Similarity Probability Contrastive Loss (JSPCL) and Stepwise Label Contrastive Loss (SLCL)
  arXiv  Detail & Related papers  (2022-12-01T15:00:16Z)
• Contrastive Bayesian Analysis for Deep Metric Learning [30.21464199249958]
  We develop a contrastive Bayesian analysis to characterize and model the posterior probabilities of image labels conditioned by their features similarity. This contrastive Bayesian analysis leads to a new loss function for deep metric learning. Our experimental results and ablation studies demonstrate that the proposed contrastive Bayesian metric learning method significantly improves the performance of deep metric learning.
  arXiv  Detail & Related papers  (2022-10-10T02:24:21Z)
• Rethinking Clustering-Based Pseudo-Labeling for Unsupervised Meta-Learning [146.11600461034746]
  Method for unsupervised meta-learning, CACTUs, is a clustering-based approach with pseudo-labeling. This approach is model-agnostic and can be combined with supervised algorithms to learn from unlabeled data. We prove that the core reason for this is lack of a clustering-friendly property in the embedding space.
  arXiv  Detail & Related papers  (2022-09-27T19:04:36Z)
• PercentMatch: Percentile-based Dynamic Thresholding for Multi-Label Semi-Supervised Classification [64.39761523935613]
  We propose a percentile-based threshold adjusting scheme to dynamically alter the score thresholds of positive and negative pseudo-labels for each class during the training. We achieve strong performance on Pascal VOC2007 and MS-COCO datasets when compared to recent SSL methods.
  arXiv  Detail & Related papers  (2022-08-30T01:27:48Z)
• Variational Distillation for Multi-View Learning [104.17551354374821]
  We design several variational information bottlenecks to exploit two key characteristics for multi-view representation learning. Under rigorously theoretical guarantee, our approach enables IB to grasp the intrinsic correlation between observations and semantic labels.
  arXiv  Detail & Related papers  (2022-06-20T03:09:46Z)
• On Modality Bias Recognition and Reduction [70.69194431713825]
  We study the modality bias problem in the context of multi-modal classification. We propose a plug-and-play loss function method, whereby the feature space for each label is adaptively learned. Our method yields remarkable performance improvements compared with the baselines.
  arXiv  Detail & Related papers  (2022-02-25T13:47:09Z)
• Incremental False Negative Detection for Contrastive Learning [95.68120675114878]
  We introduce a novel incremental false negative detection for self-supervised contrastive learning. During contrastive learning, we discuss two strategies to explicitly remove the detected false negatives. Our proposed method outperforms other self-supervised contrastive learning frameworks on multiple benchmarks within a limited compute.
  arXiv  Detail & Related papers  (2021-06-07T15:29:14Z)
• Disentangling Sampling and Labeling Bias for Learning in Large-Output Spaces [64.23172847182109]
  We show that different negative sampling schemes implicitly trade-off performance on dominant versus rare labels. We provide a unified means to explicitly tackle both sampling bias, arising from working with a subset of all labels, and labeling bias, which is inherent to the data due to label imbalance.
  arXiv  Detail & Related papers  (2021-05-12T15:40:13Z)
• Minimax Active Learning [61.729667575374606]
  Active learning aims to develop label-efficient algorithms by querying the most representative samples to be labeled by a human annotator. Current active learning techniques either rely on model uncertainty to select the most uncertain samples or use clustering or reconstruction to choose the most diverse set of unlabeled examples. We develop a semi-supervised minimax entropy-based active learning algorithm that leverages both uncertainty and diversity in an adversarial manner.
  arXiv  Detail & Related papers  (2020-12-18T19:03:40Z)
• Exploiting Sample Uncertainty for Domain Adaptive Person Re-Identification [137.9939571408506]
  We estimate and exploit the credibility of the assigned pseudo-label of each sample to alleviate the influence of noisy labels. Our uncertainty-guided optimization brings significant improvement and achieves the state-of-the-art performance on benchmark datasets.
  arXiv  Detail & Related papers  (2020-12-16T04:09:04Z)
• Learning from Aggregate Observations [82.44304647051243]
  We study the problem of learning from aggregate observations where supervision signals are given to sets of instances. We present a general probabilistic framework that accommodates a variety of aggregate observations. Simple maximum likelihood solutions can be applied to various differentiable models.
  arXiv  Detail & Related papers  (2020-04-14T06:18:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.