Improving Semi-Supervised Semantic Segmentation with Dual-Level Siamese Structure Network

• URL: http://arxiv.org/abs/2307.13938v2
• Date: Tue, 16 Apr 2024 22:29:17 GMT
• Title: Improving Semi-Supervised Semantic Segmentation with Dual-Level Siamese Structure Network
• Authors: Zhibo Tain, Xiaolin Zhang, Peng Zhang, Kun Zhan,
• Abstract summary: We propose a dual-level Siamese structure network (DSSN) for pixel-wise contrastive learning. We introduce a novel class-aware pseudo-label selection strategy for weak-to-strong supervision. Our proposed method achieves state-of-the-art results on two datasets.
• Score: 7.438140196173472
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Semi-supervised semantic segmentation (SSS) is an important task that utilizes both labeled and unlabeled data to reduce expenses on labeling training examples. However, the effectiveness of SSS algorithms is limited by the difficulty of fully exploiting the potential of unlabeled data. To address this, we propose a dual-level Siamese structure network (DSSN) for pixel-wise contrastive learning. By aligning positive pairs with a pixel-wise contrastive loss using strong augmented views in both low-level image space and high-level feature space, the proposed DSSN is designed to maximize the utilization of available unlabeled data. Additionally, we introduce a novel class-aware pseudo-label selection strategy for weak-to-strong supervision, which addresses the limitations of most existing methods that do not perform selection or apply a predefined threshold for all classes. Specifically, our strategy selects the top high-confidence prediction of the weak view for each class to generate pseudo labels that supervise the strong augmented views. This strategy is capable of taking into account the class imbalance and improving the performance of long-tailed classes. Our proposed method achieves state-of-the-art results on two datasets, PASCAL VOC 2012 and Cityscapes, outperforming other SSS algorithms by a significant margin. The source code is available at https://github.com/kunzhan/DSSN.

Related papers

• Roll With the Punches: Expansion and Shrinkage of Soft Label Selection for Semi-supervised Fine-Grained Learning [42.71454054383897]
  We propose Soft Label Selection with Confidence-Aware Clustering based on Class Transition Tracking (SoC) Our approach demonstrates its superior performance in SS-FGVC.
  arXiv  Detail & Related papers  (2023-12-19T15:22:37Z)
• SemiReward: A General Reward Model for Semi-supervised Learning [58.47299780978101]
  Semi-supervised learning (SSL) has witnessed great progress with various improvements in the self-training framework with pseudo labeling. Main challenge is how to distinguish high-quality pseudo labels against the confirmation bias. We propose a Semi-supervised Reward framework (SemiReward) that predicts reward scores to evaluate and filter out high-quality pseudo labels.
  arXiv  Detail & Related papers  (2023-10-04T17:56:41Z)
• ProtoCon: Pseudo-label Refinement via Online Clustering and Prototypical Consistency for Efficient Semi-supervised Learning [60.57998388590556]
  ProtoCon is a novel method for confidence-based pseudo-labeling. Online nature of ProtoCon allows it to utilise the label history of the entire dataset in one training cycle. It delivers significant gains and faster convergence over state-of-the-art datasets.
  arXiv  Detail & Related papers  (2023-03-22T23:51:54Z)
• Foundation Model Drives Weakly Incremental Learning for Semantic Segmentation [12.362400851574872]
  Weakly incremental learning for semantic segmentation (WILSS) is a novel and attractive task. We propose a novel and data-efficient framework for WILSS, named FMWISS.
  arXiv  Detail & Related papers  (2023-02-28T02:21:42Z)
• Transductive CLIP with Class-Conditional Contrastive Learning [68.51078382124331]
  We propose Transductive CLIP, a novel framework for learning a classification network with noisy labels from scratch. A class-conditional contrastive learning mechanism is proposed to mitigate the reliance on pseudo labels. ensemble labels is adopted as a pseudo label updating strategy to stabilize the training of deep neural networks with noisy labels.
  arXiv  Detail & Related papers  (2022-06-13T14:04:57Z)
• Learning Self-Supervised Low-Rank Network for Single-Stage Weakly and Semi-Supervised Semantic Segmentation [119.009033745244]
  This paper presents a Self-supervised Low-Rank Network ( SLRNet) for single-stage weakly supervised semantic segmentation (WSSS) and semi-supervised semantic segmentation (SSSS) SLRNet uses cross-view self-supervision, that is, it simultaneously predicts several attentive LR representations from different views of an image to learn precise pseudo-labels. Experiments on the Pascal VOC 2012, COCO, and L2ID datasets demonstrate that our SLRNet outperforms both state-of-the-art WSSS and SSSS methods with a variety of different settings.
  arXiv  Detail & Related papers  (2022-03-19T09:19:55Z)
• Isometric Propagation Network for Generalized Zero-shot Learning [72.02404519815663]
  A popular strategy is to learn a mapping between the semantic space of class attributes and the visual space of images based on the seen classes and their data. We propose Isometric propagation Network (IPN), which learns to strengthen the relation between classes within each space and align the class dependency in the two spaces. IPN achieves state-of-the-art performance on three popular Zero-shot learning benchmarks.
  arXiv  Detail & Related papers  (2021-02-03T12:45:38Z)
• Contrastive Learning for Label-Efficient Semantic Segmentation [44.10416030868873]
  Convolutional Neural Network (CNN) based semantic segmentation approaches have achieved impressive results by using large amounts of labeled data. Deep CNNs trained with the de facto cross-entropy loss can easily overfit to small amounts of labeled data. We propose a simple and effective contrastive learning-based training strategy in which we first pretrain the network using a pixel-wise, label-based contrastive loss.
  arXiv  Detail & Related papers  (2020-12-13T07:05:39Z)
• PseudoSeg: Designing Pseudo Labels for Semantic Segmentation [78.35515004654553]
  We present a re-design of pseudo-labeling to generate structured pseudo labels for training with unlabeled or weakly-labeled data. We demonstrate the effectiveness of the proposed pseudo-labeling strategy in both low-data and high-data regimes.
  arXiv  Detail & Related papers  (2020-10-19T17:59:30Z)
• Reinforced active learning for image segmentation [34.096237671643145]
  We present a new active learning strategy for semantic segmentation based on deep reinforcement learning (RL) An agent learns a policy to select a subset of small informative image regions -- opposed to entire images -- to be labeled from a pool of unlabeled data. Our method proposes a new modification of the deep Q-network (DQN) formulation for active learning, adapting it to the large-scale nature of semantic segmentation problems.
  arXiv  Detail & Related papers  (2020-02-16T14:03:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.