Top-K Pooling with Patch Contrastive Learning for Weakly-Supervised Semantic Segmentation

• URL: http://arxiv.org/abs/2310.09828v2
• Date: Wed, 27 Dec 2023 11:40:27 GMT
• Title: Top-K Pooling with Patch Contrastive Learning for Weakly-Supervised Semantic Segmentation
• Authors: Wangyu Wu, Tianhong Dai, Xiaowei Huang, Fei Ma, Jimin Xiao
• Abstract summary: We introduce a novel ViT-based WSSS method named top-K pooling with patch contrastive learning (TKP-PCL) A patch contrastive error (PCE) is also proposed to enhance the patch embeddings to further improve the final results. Our approach is very efficient and outperforms other state-of-the-art WSSS methods on the PASCAL 2012 dataset.
• Score: 25.628382644404066
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Weakly Supervised Semantic Segmentation (WSSS) using only image-level labels has gained significant attention due to cost-effectiveness. Recently, Vision Transformer (ViT) based methods without class activation map (CAM) have shown greater capability in generating reliable pseudo labels than previous methods using CAM. However, the current ViT-based methods utilize max pooling to select the patch with the highest prediction score to map the patch-level classification to the image-level one, which may affect the quality of pseudo labels due to the inaccurate classification of the patches. In this paper, we introduce a novel ViT-based WSSS method named top-K pooling with patch contrastive learning (TKP-PCL), which employs a top-K pooling layer to alleviate the limitations of previous max pooling selection. A patch contrastive error (PCE) is also proposed to enhance the patch embeddings to further improve the final results. The experimental results show that our approach is very efficient and outperforms other state-of-the-art WSSS methods on the PASCAL VOC 2012 dataset.

Related papers

• PEPL: Precision-Enhanced Pseudo-Labeling for Fine-Grained Image Classification in Semi-Supervised Learning [3.801446153948012]
  We introduce Precision-Enhanced Pseudo-Labeling(PEPL) approach for fine-grained image classification within a semi-supervised learning framework. Our method leverages the abundance of unlabeled data by generating high-quality pseudo-labels. We achieve state-of-the-art performance on benchmark datasets, demonstrating significant improvements over existing semi-supervised strategies.
  arXiv  Detail & Related papers  (2024-09-05T02:32:07Z)
• APC: Adaptive Patch Contrast for Weakly Supervised Semantic Segmentation [22.808117374130198]
  Weakly Supervised Semantic dataset (WSSS) using only image-level labels has gained significant attention due to its cost-effectiveness. Recent methods based on Vision Transformers (ViT) have demonstrated superior capabilities in generating reliable pseudo-labels. We introduce a novel ViT-based WSSS method named textit Patch Contrast ( APC) that significantly enhances patch embedding learning.
  arXiv  Detail & Related papers  (2024-07-15T12:10:05Z)
• Adapting Conformal Prediction to Distribution Shifts Without Labels [16.478151550456804]
  Conformal prediction (CP) enables machine learning models to output prediction sets with guaranteed coverage rate. Our goal is to improve the quality of CP-generated prediction sets using only unlabeled data from the test domain. This is achieved by two new methods called ECP and EACP, that adjust the score function in CP according to the base model's uncertainty on the unlabeled test data.
  arXiv  Detail & Related papers  (2024-06-03T15:16:02Z)
• Active Label Correction for Semantic Segmentation with Foundation Models [34.0733215363568]
  We propose an effective framework of active label correction (ALC) based on a design of correction query to rectify pseudo labels of pixels. Our method comprises two key techniques: (i) an annotator-friendly design of correction query with the pseudo labels, and (ii) an acquisition function looking ahead label expansions based on the superpixels. Experimental results on PASCAL, Cityscapes, and Kvasir-SEG datasets demonstrate the effectiveness of our ALC framework.
  arXiv  Detail & Related papers  (2024-03-16T06:10:22Z)
• One-Step Multi-View Clustering Based on Transition Probability [61.841829428397034]
  We introduce the One-Step Multi-View Clustering Based on Transition Probability (OSMVC-TP) Our method directly learns the transition probabilities from anchor points to categories, and calculates the transition probabilities from samples to categories. To maintain consistency in labels across different views, we apply a Schatten p-norm constraint on the tensor composed of the soft labels.
  arXiv  Detail & Related papers  (2024-03-03T09:43:23Z)
• Exploiting Low-confidence Pseudo-labels for Source-free Object Detection [54.98300313452037]
  Source-free object detection (SFOD) aims to adapt a source-trained detector to an unlabeled target domain without access to the labeled source data. Current SFOD methods utilize a threshold-based pseudo-label approach in the adaptation phase. We propose a new approach to take full advantage of pseudo-labels by introducing high and low confidence thresholds.
  arXiv  Detail & Related papers  (2023-10-19T12:59:55Z)
• ProTeCt: Prompt Tuning for Taxonomic Open Set Classification [59.59442518849203]
  Few-shot adaptation methods do not fare well in the taxonomic open set (TOS) setting. We propose a prompt tuning technique that calibrates the hierarchical consistency of model predictions. A new Prompt Tuning for Hierarchical Consistency (ProTeCt) technique is then proposed to calibrate classification across label set granularities.
  arXiv  Detail & Related papers  (2023-06-04T02:55:25Z)
• Dense FixMatch: a simple semi-supervised learning method for pixel-wise prediction tasks [68.36996813591425]
  We propose Dense FixMatch, a simple method for online semi-supervised learning of dense and structured prediction tasks. We enable the application of FixMatch in semi-supervised learning problems beyond image classification by adding a matching operation on the pseudo-labels. Dense FixMatch significantly improves results compared to supervised learning using only labeled data, approaching its performance with 1/4 of the labeled samples.
  arXiv  Detail & Related papers  (2022-10-18T15:02:51Z)
• Rethinking Pseudo Labels for Semi-Supervised Object Detection [84.697097472401]
  We introduce certainty-aware pseudo labels tailored for object detection. We dynamically adjust the thresholds used to generate pseudo labels and reweight loss functions for each category to alleviate the class imbalance problem. Our approach improves supervised baselines by up to 10% AP using only 1-10% labeled data from COCO.
  arXiv  Detail & Related papers  (2021-06-01T01:32:03Z)
• In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning [53.1047775185362]
  Pseudo-labeling (PL) is a general SSL approach that does not have this constraint but performs relatively poorly in its original formulation. We argue that PL underperforms due to the erroneous high confidence predictions from poorly calibrated models. We propose an uncertainty-aware pseudo-label selection (UPS) framework which improves pseudo labeling accuracy by drastically reducing the amount of noise encountered in the training process.
  arXiv  Detail & Related papers  (2021-01-15T23:29:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.