Weakly Supervised Semantic Segmentation via Progressive Patch Learning

• URL: http://arxiv.org/abs/2209.07828v1
• Date: Fri, 16 Sep 2022 09:54:17 GMT
• Title: Weakly Supervised Semantic Segmentation via Progressive Patch Learning
• Authors: Jinlong Li, Zequn Jie, Xu Wang, Yu Zhou, Xiaolin Wei, Lin Ma
• Abstract summary: "Progressive Patch Learning" approach is proposed to improve the local details extraction of the classification. "Patch Learning" destructs the feature maps into patches and independently processes each local patch in parallel before the final aggregation. "Progressive Patch Learning" further extends the feature destruction and patch learning to multi-level granularities in a progressive manner.
• Score: 39.87150496277798
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Most of the existing semantic segmentation approaches with image-level class labels as supervision, highly rely on the initial class activation map (CAM) generated from the standard classification network. In this paper, a novel "Progressive Patch Learning" approach is proposed to improve the local details extraction of the classification, producing the CAM better covering the whole object rather than only the most discriminative regions as in CAMs obtained in conventional classification models. "Patch Learning" destructs the feature maps into patches and independently processes each local patch in parallel before the final aggregation. Such a mechanism enforces the network to find weak information from the scattered discriminative local parts, achieving enhanced local details sensitivity. "Progressive Patch Learning" further extends the feature destruction and patch learning to multi-level granularities in a progressive manner. Cooperating with a multi-stage optimization strategy, such a "Progressive Patch Learning" mechanism implicitly provides the model with the feature extraction ability across different locality-granularities. As an alternative to the implicit multi-granularity progressive fusion approach, we additionally propose an explicit method to simultaneously fuse features from different granularities in a single model, further enhancing the CAM quality on the full object coverage. Our proposed method achieves outstanding performance on the PASCAL VOC 2012 dataset e.g., with 69.6$% mIoU on the test set), which surpasses most existing weakly supervised semantic segmentation methods. Code will be made publicly available here https://github.com/TyroneLi/PPL_WSSS.

Related papers

• Generalize or Detect? Towards Robust Semantic Segmentation Under Multiple Distribution Shifts [56.57141696245328]
  In open-world scenarios, where both novel classes and domains may exist, an ideal segmentation model should detect anomaly classes for safety. Existing methods often struggle to distinguish between domain-level and semantic-level distribution shifts.
  arXiv  Detail & Related papers  (2024-11-06T11:03:02Z)
• Beyond Pixels: Semi-Supervised Semantic Segmentation with a Multi-scale Patch-based Multi-Label Classifier [37.02049053586457]
  We introduce Multi-scale Patch-based Multi-label (MPMC) MPMC offers patch-level supervision, enabling the discrimination of pixel regions of different classes within a patch. MPMC learns an adaptive pseudo-label weight, using patch-level classification to alleviate the impact of the teacher's noisy pseudo-label supervision.
  arXiv  Detail & Related papers  (2024-07-04T16:21:29Z)
• Boundary-Refined Prototype Generation: A General End-to-End Paradigm for Semi-Supervised Semantic Segmentation [23.00156170789867]
  Semi-supervised semantic segmentation has attracted increasing attention in computer vision. Current approaches isolate prototype generation from the main training framework. We propose a novel end-to-end boundary-refined prototype generation (BRPG) method.
  arXiv  Detail & Related papers  (2023-07-19T16:12:37Z)
• Semantics-Aware Dynamic Localization and Refinement for Referring Image Segmentation [102.25240608024063]
  Referring image segments an image from a language expression. We develop an algorithm that shifts from being localization-centric to segmentation-language. Compared to its counterparts, our method is more versatile yet effective.
  arXiv  Detail & Related papers  (2023-03-11T08:42:40Z)
• Saliency Guided Inter- and Intra-Class Relation Constraints for Weakly Supervised Semantic Segmentation [66.87777732230884]
  We propose a saliency guided Inter- and Intra-Class Relation Constrained (I$2$CRC) framework to assist the expansion of the activated object regions. We also introduce an object guided label refinement module to take a full use of both the segmentation prediction and the initial labels for obtaining superior pseudo-labels.
  arXiv  Detail & Related papers  (2022-06-20T03:40:56Z)
• Beyond the Prototype: Divide-and-conquer Proxies for Few-shot Segmentation [63.910211095033596]
  Few-shot segmentation aims to segment unseen-class objects given only a handful of densely labeled samples. We propose a simple yet versatile framework in the spirit of divide-and-conquer. Our proposed approach, named divide-and-conquer proxies (DCP), allows for the development of appropriate and reliable information.
  arXiv  Detail & Related papers  (2022-04-21T06:21:14Z)
• Beyond Simple Meta-Learning: Multi-Purpose Models for Multi-Domain, Active and Continual Few-Shot Learning [41.07029317930986]
  We propose a variance-sensitive class of models that operates in a low-label regime. The first method, Simple CNAPS, employs a hierarchically regularized Mahalanobis-distance based classifier. We further extend this approach to a transductive learning setting, proposing Transductive CNAPS.
  arXiv  Detail & Related papers  (2022-01-13T18:59:02Z)
• Learning Semantically Enhanced Feature for Fine-Grained Image Classification [27.136912902584093]
  Our approach learns fine-grained features by enhancing the semantics of sub-features of a global feature. Our approach is parameter parsimonious and can be easily integrated into the backbone model as a plug-and-play module for end-to-end training.
  arXiv  Detail & Related papers  (2020-06-24T03:41:12Z)
• Fine-Grained Visual Classification via Progressive Multi-Granularity Training of Jigsaw Patches [67.51747235117]
  Fine-grained visual classification (FGVC) is much more challenging than traditional classification tasks. Recent works mainly tackle this problem by focusing on how to locate the most discriminative parts. We propose a novel framework for fine-grained visual classification to tackle these problems.
  arXiv  Detail & Related papers  (2020-03-08T19:27:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.