DHR: Dual Features-Driven Hierarchical Rebalancing in Inter- and Intra-Class Regions for Weakly-Supervised Semantic Segmentation

• URL: http://arxiv.org/abs/2404.00380v2
• Date: Sun, 19 May 2024 14:27:40 GMT
• Title: DHR: Dual Features-Driven Hierarchical Rebalancing in Inter- and Intra-Class Regions for Weakly-Supervised Semantic Segmentation
• Abstract summary: Weakly-supervised semantic segmentation (WSS) ensures high-quality segmentation with limited data. WSS faces challenges related to minor classes since those are overlooked in images with adjacent multiple classes. We first address this by employing unsupervised and weakly-supervised feature maps instead of conventional methodologies.
• Score: 7.545489807578403
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Weakly-supervised semantic segmentation (WSS) ensures high-quality segmentation with limited data and excels when employed as input seed masks for large-scale vision models such as Segment Anything. However, WSS faces challenges related to minor classes since those are overlooked in images with adjacent multiple classes, a limitation originating from the overfitting of traditional expansion methods like Random Walk. We first address this by employing unsupervised and weakly-supervised feature maps instead of conventional methodologies, allowing for hierarchical mask enhancement. This method distinctly categorizes higher-level classes and subsequently separates their associated lower-level classes, ensuring all classes are correctly restored in the mask without losing minor ones. Our approach, validated through extensive experimentation, significantly improves WSS across five benchmarks (VOC: 79.8\%, COCO: 53.9\%, Context: 49.0\%, ADE: 32.9\%, Stuff: 37.4\%), reducing the gap with fully supervised methods by over 84\% on the VOC validation set. Code is available at https://github.com/shjo-april/DHR.

Related papers

• Training-Free Fine-Grained Semantic Segmentations in Low Data Regimes: A FungiTastic Baseline [2.4388965927178767]
  Fine-grained semantic segmentation requires both precise localization and discrimination between visually similar classes.<n>In FungiTastic, this problem is further complicated by a long-tailed distribution and strong variation in image acquisition conditions.<n>We propose a training-free two-stage framework that decouples segmentation from classification.
  arXiv  Detail & Related papers  (2026-05-21T13:44:52Z)
• TagCLIP: A Local-to-Global Framework to Enhance Open-Vocabulary Multi-Label Classification of CLIP Without Training [29.431698321195814]
  Contrastive Language-Image Pre-training (CLIP) has demonstrated impressive capabilities in open-vocabulary classification. CLIP shows poor performance on multi-label datasets because the global feature tends to be dominated by the most prominent class. We propose a local-to-global framework to obtain image tags.
  arXiv  Detail & Related papers  (2023-12-20T08:15:40Z)
• High-fidelity Pseudo-labels for Boosting Weakly-Supervised Segmentation [17.804090651425955]
  Image-level weakly-supervised segmentation (WSSS) reduces the usually vast data annotation cost by surrogate segmentation masks during training. Our work is based on two techniques for improving CAMs; importance sampling, which is a substitute for GAP, and the feature similarity loss. We reformulate both techniques based on binomial posteriors of multiple independent binary problems. This has two benefits; their performance is improved and they become more general, resulting in an add-on method that can boost virtually any WSSS method.
  arXiv  Detail & Related papers  (2023-04-05T17:43:57Z)
• The Devil is in the Points: Weakly Semi-Supervised Instance Segmentation via Point-Guided Mask Representation [61.027468209465354]
  We introduce a novel learning scheme named weakly semi-supervised instance segmentation (WSSIS) with point labels. We propose a method for WSSIS that can effectively leverage the budget-friendly point labels as a powerful weak supervision source. We conduct extensive experiments on COCO and BDD100K datasets, and the proposed method achieves promising results comparable to those of the fully-supervised model.
  arXiv  Detail & Related papers  (2023-03-27T10:11:22Z)
• Foundation Model Drives Weakly Incremental Learning for Semantic Segmentation [12.362400851574872]
  Weakly incremental learning for semantic segmentation (WILSS) is a novel and attractive task. We propose a novel and data-efficient framework for WILSS, named FMWISS.
  arXiv  Detail & Related papers  (2023-02-28T02:21:42Z)
• Masked Unsupervised Self-training for Zero-shot Image Classification [98.23094305347709]
  Masked Unsupervised Self-Training (MUST) is a new approach which leverages two different and complimentary sources of supervision: pseudo-labels and raw images. MUST improves upon CLIP by a large margin and narrows the performance gap between unsupervised and supervised classification.
  arXiv  Detail & Related papers  (2022-06-07T02:03:06Z)
• Per-Pixel Classification is Not All You Need for Semantic Segmentation [184.2905747595058]
  Mask classification is sufficiently general to solve both semantic- and instance-level segmentation tasks. We propose MaskFormer, a simple mask classification model which predicts a set of binary masks. Our method outperforms both current state-of-the-art semantic (55.6 mIoU on ADE20K) and panoptic segmentation (52.7 PQ on COCO) models.
  arXiv  Detail & Related papers  (2021-07-13T17:59:50Z)
• SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption [72.35532598131176]
  We propose SCARF, a technique for contrastive learning, where views are formed by corrupting a random subset of features. We show that SCARF complements existing strategies and outperforms alternatives like autoencoders.
  arXiv  Detail & Related papers  (2021-06-29T08:08:33Z)
• Adaptive Class Suppression Loss for Long-Tail Object Detection [49.7273558444966]
  We devise a novel Adaptive Class Suppression Loss (ACSL) to improve the detection performance of tail categories. Our ACSL achieves 5.18% and 5.2% improvements with ResNet50-FPN, and sets a new state of the art.
  arXiv  Detail & Related papers  (2021-04-02T05:12:31Z)
• Prior Guided Feature Enrichment Network for Few-Shot Segmentation [64.91560451900125]
  State-of-the-art semantic segmentation methods require sufficient labeled data to achieve good results. Few-shot segmentation is proposed to tackle this problem by learning a model that quickly adapts to new classes with a few labeled support samples. Theses frameworks still face the challenge of generalization ability reduction on unseen classes due to inappropriate use of high-level semantic information.
  arXiv  Detail & Related papers  (2020-08-04T10:41:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.