Improving Semi-Supervised Contrastive Learning via Entropy-Weighted Confidence Integration of Anchor-Positive Pairs

• URL: http://arxiv.org/abs/2601.04555v1
• Date: Thu, 08 Jan 2026 03:34:08 GMT
• Title: Improving Semi-Supervised Contrastive Learning via Entropy-Weighted Confidence Integration of Anchor-Positive Pairs
• Authors: Shogo Nakayama, Masahiro Okuda,
• Abstract summary: We propose a novel loss function that estimates the confidence of each sample based on the entropy of its predicted probability distribution.<n> Experimental results demonstrate that the proposed method improves classification accuracy and achieves more stable learning performance even under low-label conditions.
• Score: 1.0312968200748116
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Conventional semi-supervised contrastive learning methods assign pseudo-labels only to samples whose highest predicted class probability exceeds a predefined threshold, and then perform supervised contrastive learning using those selected samples. In this study, we propose a novel loss function that estimates the confidence of each sample based on the entropy of its predicted probability distribution and applies confidence-based adaptive weighting. This approach enables pseudo-label assignment even to samples that were previously excluded from training and facilitates contrastive learning that accounts for the confidence of both anchor and positive samples in a more principled manner. Experimental results demonstrate that the proposed method improves classification accuracy and achieves more stable learning performance even under low-label conditions.

Related papers

• TRUST: Test-time Resource Utilization for Superior Trustworthiness [15.031121920821109]
  We propose a novel test-time optimization method that accounts for the impact of such noise to produce more reliable confidence estimates.<n>This score defines a monotonic subset-selection function, where population accuracy consistently increases as samples with lower scores are removed.
  arXiv  Detail & Related papers  (2025-06-06T12:52:32Z)
• Self-Knowledge Distillation for Learning Ambiguity [11.755814660833549]
  Recent language models often over-confidently predict a single label without consideration for its correctness. We propose a novel self-knowledge distillation method that enables models to learn label distributions more accurately. We validate our method on diverse NLU benchmark datasets and the experimental results demonstrate its effectiveness in producing better label distributions.
  arXiv  Detail & Related papers  (2024-06-14T05:11:32Z)
• Binary Classification with Confidence Difference [100.08818204756093]
  This paper delves into a novel weakly supervised binary classification problem called confidence-difference (ConfDiff) classification. We propose a risk-consistent approach to tackle this problem and show that the estimation error bound the optimal convergence rate. We also introduce a risk correction approach to mitigate overfitting problems, whose consistency and convergence rate are also proven.
  arXiv  Detail & Related papers  (2023-10-09T11:44:50Z)
• Shrinking Class Space for Enhanced Certainty in Semi-Supervised Learning [59.44422468242455]
  We propose a novel method dubbed ShrinkMatch to learn uncertain samples. For each uncertain sample, it adaptively seeks a shrunk class space, which merely contains the original top-1 class. We then impose a consistency regularization between a pair of strongly and weakly augmented samples in the shrunk space to strive for discriminative representations.
  arXiv  Detail & Related papers  (2023-08-13T14:05:24Z)
• Calibrating Deep Neural Networks using Explicit Regularisation and Dynamic Data Pruning [25.982037837953268]
  Deep neural networks (DNN) are prone to miscalibrated predictions, often exhibiting a mismatch between the predicted output and the associated confidence scores. We propose a novel regularization technique that can be used with classification losses, leading to state-of-the-art calibrated predictions at test time.
  arXiv  Detail & Related papers  (2022-12-20T05:34:58Z)
• ProBoost: a Boosting Method for Probabilistic Classifiers [55.970609838687864]
  ProBoost is a new boosting algorithm for probabilistic classifiers. It uses the uncertainty of each training sample to determine the most challenging/uncertain ones. It produces a sequence that progressively focuses on the samples found to have the highest uncertainty.
  arXiv  Detail & Related papers  (2022-09-04T12:49:20Z)
• RegMixup: Mixup as a Regularizer Can Surprisingly Improve Accuracy and Out Distribution Robustness [94.69774317059122]
  We show that the effectiveness of the well celebrated Mixup can be further improved if instead of using it as the sole learning objective, it is utilized as an additional regularizer to the standard cross-entropy loss. This simple change not only provides much improved accuracy but also significantly improves the quality of the predictive uncertainty estimation of Mixup.
  arXiv  Detail & Related papers  (2022-06-29T09:44:33Z)
• Confidence Adaptive Regularization for Deep Learning with Noisy Labels [2.0349696181833337]
  Recent studies on the memorization effects of deep neural networks on noisy labels show that the networks first fit the correctly-labeled training samples before memorizing the mislabeled samples. Motivated by this early-learning phenomenon, we propose a novel method to prevent memorization of the mislabeled samples. We provide the theoretical analysis and conduct the experiments on synthetic and real-world datasets, demonstrating that our approach achieves comparable results to the state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-08-18T15:51:25Z)
• Learning from Similarity-Confidence Data [94.94650350944377]
  We investigate a novel weakly supervised learning problem of learning from similarity-confidence (Sconf) data. We propose an unbiased estimator of the classification risk that can be calculated from only Sconf data and show that the estimation error bound achieves the optimal convergence rate.
  arXiv  Detail & Related papers  (2021-02-13T07:31:16Z)
• Binary Classification from Positive Data with Skewed Confidence [85.18941440826309]
  Positive-confidence (Pconf) classification is a promising weakly-supervised learning method. In practice, the confidence may be skewed by bias arising in an annotation process. We introduce the parameterized model of the skewed confidence, and propose the method for selecting the hyper parameter.
  arXiv  Detail & Related papers  (2020-01-29T00:04:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.