WeakSupCon: Weakly Supervised Contrastive Learning for Encoder Pre-training

• URL: http://arxiv.org/abs/2503.04165v2
• Date: Wed, 30 Jul 2025 23:37:48 GMT
• Title: WeakSupCon: Weakly Supervised Contrastive Learning for Encoder Pre-training
• Authors: Bodong Zhang, Hamid Manoochehri, Xiwen Li, Beatrice S. Knudsen, Tolga Tasdizen,
• Abstract summary: We propose a novel weakly supervised feature representation learning method called Weakly Supervised Contrastive Learning (WeakSupCon)<n>In our method, we employ multi-task learning and define distinct contrastive losses for samples with different bag labels.<n>Our experiments demonstrate that the features generated using WeakSupCon with limited computing resources significantly enhance MIL classification performance.
• Score: 1.2233362977312943
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Weakly supervised multiple instance learning (MIL) is a challenging task given that only bag-level labels are provided, while each bag typically contains multiple instances. This topic has been extensively studied in histopathological image analysis, where labels are usually available only at the whole slide image (WSI) level, while each WSI could be divided into thousands of small image patches for training. The dominant MIL approaches focus on feature aggregation and take fixed patch features as inputs. However, weakly supervised feature representation learning in MIL settings is always neglected. Those features used to be generated by self-supervised learning methods that do not utilize weak labels, or by foundation encoders pre-trained on other large datasets. In this paper, we propose a novel weakly supervised feature representation learning method called Weakly Supervised Contrastive Learning (WeakSupCon) that utilizes bag-level labels. In our method, we employ multi-task learning and define distinct contrastive losses for samples with different bag labels. Our experiments demonstrate that the features generated using WeakSupCon with limited computing resources significantly enhance MIL classification performance compared to self-supervised approaches across three datasets. Our WeakSupCon code is available at github.com/BzhangURU/Paper_WeakSupCon

Related papers

• ViLa-MIL: Dual-scale Vision-Language Multiple Instance Learning for Whole Slide Image Classification [52.405499816861635]
  Multiple instance learning (MIL)-based framework has become the mainstream for processing the whole slide image (WSI) We propose a dual-scale vision-language multiple instance learning (ViLa-MIL) framework for whole slide image classification.
  arXiv  Detail & Related papers  (2025-02-12T13:28:46Z)
• MixSup: Mixed-grained Supervision for Label-efficient LiDAR-based 3D Object Detection [59.1417156002086]
  MixSup is a more practical paradigm simultaneously utilizing massive cheap coarse labels and a limited number of accurate labels for Mixed-grained Supervision. MixSup achieves up to 97.31% of fully supervised performance, using cheap cluster annotations and only 10% box annotations.
  arXiv  Detail & Related papers  (2024-01-29T17:05:19Z)
• Virtual Category Learning: A Semi-Supervised Learning Method for Dense Prediction with Extremely Limited Labels [63.16824565919966]
  This paper proposes to use confusing samples proactively without label correction. A Virtual Category (VC) is assigned to each confusing sample in such a way that it can safely contribute to the model optimisation. Our intriguing findings highlight the usage of VC learning in dense vision tasks.
  arXiv  Detail & Related papers  (2023-12-02T16:23:52Z)
• Rethinking Multiple Instance Learning for Whole Slide Image Classification: A Bag-Level Classifier is a Good Instance-Level Teacher [22.080213609228547]
  Multiple Instance Learning has demonstrated promise in Whole Slide Image (WSI) classification. Existing methods generally adopt a two-stage approach, comprising a non-learnable feature embedding stage and a classifier training stage. We propose that a bag-level classifier can be a good instance-level teacher.
  arXiv  Detail & Related papers  (2023-12-02T10:16:03Z)
• Rethinking Multiple Instance Learning for Whole Slide Image Classification: A Good Instance Classifier is All You Need [18.832471712088353]
  We propose an instance-level weakly supervised contrastive learning algorithm for the first time under the MIL setting. We also propose an accurate pseudo label generation method through prototype learning.
  arXiv  Detail & Related papers  (2023-07-05T12:44:52Z)
• The Rise of AI Language Pathologists: Exploring Two-level Prompt Learning for Few-shot Weakly-supervised Whole Slide Image Classification [23.004237397025822]
  This paper introduces the novel concept of few-shot weakly supervised learning for pathology Whole Slide Image (WSI) classification, denoted as FSWC. A solution is proposed based on prompt learning and the utilization of a large language model, GPT-4. Our approach incorporates the utilization of GPT-4 in a question-and-answer mode to obtain language prior knowledge at both the instance and bag levels, which are then integrated into the instance and bag level language prompts.
  arXiv  Detail & Related papers  (2023-05-29T05:35:44Z)
• Multi-Level Contrastive Learning for Dense Prediction Task [59.591755258395594]
  We present Multi-Level Contrastive Learning for Dense Prediction Task (MCL), an efficient self-supervised method for learning region-level feature representation for dense prediction tasks. Our method is motivated by the three key factors in detection: localization, scale consistency and recognition. Our method consistently outperforms the recent state-of-the-art methods on various datasets with significant margins.
  arXiv  Detail & Related papers  (2023-04-04T17:59:04Z)
• MaskCon: Masked Contrastive Learning for Coarse-Labelled Dataset [19.45520684918576]
  We propose a contrastive learning method, called $textbfMask$ed $textbfCon$trastive learning($textbfMaskCon$) For each sample our method generates soft-labels with the aid of coarse labels against other samples and another augmented view of the sample in question. Our method achieves significant improvement over the current state-of-the-art in various datasets.
  arXiv  Detail & Related papers  (2023-03-22T17:08:31Z)
• Multiple Instance Learning via Iterative Self-Paced Supervised Contrastive Learning [22.07044031105496]
  Learning representations for individual instances when only bag-level labels are available is a challenge in multiple instance learning (MIL) We propose a novel framework, Iterative Self-paced Supervised Contrastive Learning for MIL Representations (ItS2CLR) It improves the learned representation by exploiting instance-level pseudo labels derived from the bag-level labels.
  arXiv  Detail & Related papers  (2022-10-17T21:43:32Z)
• Large Loss Matters in Weakly Supervised Multi-Label Classification [50.262533546999045]
  We first regard unobserved labels as negative labels, casting the W task into noisy multi-label classification. We propose novel methods for W which reject or correct the large loss samples to prevent model from memorizing the noisy label. Our methodology actually works well, validating that treating large loss properly matters in a weakly supervised multi-label classification.
  arXiv  Detail & Related papers  (2022-06-08T08:30:24Z)
• Masked Unsupervised Self-training for Zero-shot Image Classification [98.23094305347709]
  Masked Unsupervised Self-Training (MUST) is a new approach which leverages two different and complimentary sources of supervision: pseudo-labels and raw images. MUST improves upon CLIP by a large margin and narrows the performance gap between unsupervised and supervised classification.
  arXiv  Detail & Related papers  (2022-06-07T02:03:06Z)
• Large-Scale Pre-training for Person Re-identification with Noisy Labels [125.49696935852634]
  We develop a large-scale Pre-training framework utilizing Noisy Labels (PNL) In principle, joint learning of these three modules not only clusters similar examples to one prototype, but also rectifies noisy labels based on the prototype assignment. This simple pre-training task provides a scalable way to learn SOTA Re-ID representations from scratch on "LUPerson-NL" without bells and whistles.
  arXiv  Detail & Related papers  (2022-03-30T17:59:58Z)
• Learning Self-Supervised Low-Rank Network for Single-Stage Weakly and Semi-Supervised Semantic Segmentation [119.009033745244]
  This paper presents a Self-supervised Low-Rank Network ( SLRNet) for single-stage weakly supervised semantic segmentation (WSSS) and semi-supervised semantic segmentation (SSSS) SLRNet uses cross-view self-supervision, that is, it simultaneously predicts several attentive LR representations from different views of an image to learn precise pseudo-labels. Experiments on the Pascal VOC 2012, COCO, and L2ID datasets demonstrate that our SLRNet outperforms both state-of-the-art WSSS and SSSS methods with a variety of different settings.
  arXiv  Detail & Related papers  (2022-03-19T09:19:55Z)
• Multi-label Iterated Learning for Image Classification with Label Ambiguity [3.5736176624479654]
  We propose multi-label iterated learning (MILe) to incorporate the inductive biases of multi-label learning from single labels. MILe is a simple yet effective procedure that builds a multi-label description of the image by propagating binary predictions. We show that MILe is effective reducing label noise, achieving state-of-the-art performance on real-world large-scale noisy data such as WebVision.
  arXiv  Detail & Related papers  (2021-11-23T22:10:00Z)
• Nested Multiple Instance Learning with Attention Mechanisms [2.6552823781152366]
  Multiple instance learning (MIL) is a type of weakly supervised learning where multiple instances of data with unknown labels are sorted into bags. We propose Nested MIL, where only the outermost bag is labelled and instances are represented as latent labels. Our proposed model provides high accuracy performance as well as spotting relevant instances on image regions.
  arXiv  Detail & Related papers  (2021-11-01T13:41:09Z)
• Semi-weakly Supervised Contrastive Representation Learning for Retinal Fundus Images [0.2538209532048867]
  We propose a semi-weakly supervised contrastive learning framework for representation learning using semi-weakly annotated images. We empirically validate the transfer learning performance of SWCL on seven public retinal fundus datasets.
  arXiv  Detail & Related papers  (2021-08-04T15:50:09Z)
• SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption [72.35532598131176]
  We propose SCARF, a technique for contrastive learning, where views are formed by corrupting a random subset of features. We show that SCARF complements existing strategies and outperforms alternatives like autoencoders.
  arXiv  Detail & Related papers  (2021-06-29T08:08:33Z)
• Generative Multi-Label Zero-Shot Learning [136.17594611722285]
  Multi-label zero-shot learning strives to classify images into multiple unseen categories for which no data is available during training. Our work is the first to tackle the problem of multi-label feature in the (generalized) zero-shot setting. Our cross-level fusion-based generative approach outperforms the state-of-the-art on all three datasets.
  arXiv  Detail & Related papers  (2021-01-27T18:56:46Z)
• UniT: Unified Knowledge Transfer for Any-shot Object Detection and Segmentation [52.487469544343305]
  Methods for object detection and segmentation rely on large scale instance-level annotations for training. We propose an intuitive and unified semi-supervised model that is applicable to a range of supervision.
  arXiv  Detail & Related papers  (2020-06-12T22:45:47Z)
• Dual-stream Maximum Self-attention Multi-instance Learning [11.685285490589981]
  Multi-instance learning (MIL) is a form of weakly supervised learning where a single class label is assigned to a bag of instances while the instance-level labels are not available. We propose a dual-stream maximum self-attention MIL model (DSMIL) parameterized by neural networks. Our method achieves superior performance compared to the best MIL methods and demonstrates state-of-the-art performance on benchmark MIL datasets.
  arXiv  Detail & Related papers  (2020-06-09T22:44:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.