Activating the Discriminability of Novel Classes for Few-shot Segmentation

• URL: http://arxiv.org/abs/2212.01131v1
• Date: Fri, 2 Dec 2022 12:22:36 GMT
• Title: Activating the Discriminability of Novel Classes for Few-shot Segmentation
• Authors: Dianwen Mei, Wei Zhuo, Jiandong Tian, Guangming Lu, Wenjie Pei
• Abstract summary: We propose to activate the discriminability of novel classes explicitly in both the feature encoding stage and the prediction stage for segmentation. In the prediction stage for segmentation, we learn an Self-Refined Online Foreground-Background classifier (SROFB), which is able to refine itself using the high-confidence pixels of query image.
• Score: 48.542627940781095
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite the remarkable success of existing methods for few-shot segmentation, there remain two crucial challenges. First, the feature learning for novel classes is suppressed during the training on base classes in that the novel classes are always treated as background. Thus, the semantics of novel classes are not well learned. Second, most of existing methods fail to consider the underlying semantic gap between the support and the query resulting from the representative bias by the scarce support samples. To circumvent these two challenges, we propose to activate the discriminability of novel classes explicitly in both the feature encoding stage and the prediction stage for segmentation. In the feature encoding stage, we design the Semantic-Preserving Feature Learning module (SPFL) to first exploit and then retain the latent semantics contained in the whole input image, especially those in the background that belong to novel classes. In the prediction stage for segmentation, we learn an Self-Refined Online Foreground-Background classifier (SROFB), which is able to refine itself using the high-confidence pixels of query image to facilitate its adaptation to the query image and bridge the support-query semantic gap. Extensive experiments on PASCAL-5$^i$ and COCO-20$^i$ datasets demonstrates the advantages of these two novel designs both quantitatively and qualitatively.

Related papers

• Semantic Enhanced Few-shot Object Detection [37.715912401900745]
  We propose a fine-tuning based FSOD framework that utilizes semantic embeddings for better detection. Our method allows each novel class to construct a compact feature space without being confused with similar base classes.
  arXiv  Detail & Related papers  (2024-06-19T12:40:55Z)
• Harmonizing Base and Novel Classes: A Class-Contrastive Approach for Generalized Few-Shot Segmentation [78.74340676536441]
  We propose a class contrastive loss and a class relationship loss to regulate prototype updates and encourage a large distance between prototypes. Our proposed approach achieves new state-of-the-art performance for the generalized few-shot segmentation task on PASCAL VOC and MS COCO datasets.
  arXiv  Detail & Related papers  (2023-03-24T00:30:25Z)
• Automatically Discovering Novel Visual Categories with Self-supervised Prototype Learning [68.63910949916209]
  This paper tackles the problem of novel category discovery (NCD), which aims to discriminate unknown categories in large-scale image collections. We propose a novel adaptive prototype learning method consisting of two main stages: prototypical representation learning and prototypical self-training. We conduct extensive experiments on four benchmark datasets and demonstrate the effectiveness and robustness of the proposed method with state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-08-01T16:34:33Z)
• Incremental Few-Shot Learning via Implanting and Compressing [13.122771115838523]
  Incremental Few-Shot Learning requires a model to continually learn novel classes from only a few examples. We propose a two-step learning strategy referred to as textbfImplanting and textbfCompressing. Specifically, in the textbfImplanting step, we propose to mimic the data distribution of novel classes with the assistance of data-abundant base set. In the textbf step, we adapt the feature extractor to precisely represent each novel class for enhancing intra-class compactness.
  arXiv  Detail & Related papers  (2022-03-19T11:04:43Z)
• Learning What Not to Segment: A New Perspective on Few-Shot Segmentation [63.910211095033596]
  Recently few-shot segmentation (FSS) has been extensively developed. This paper proposes a fresh and straightforward insight to alleviate the problem. In light of the unique nature of the proposed approach, we also extend it to a more realistic but challenging setting.
  arXiv  Detail & Related papers  (2022-03-15T03:08:27Z)
• Modeling the Background for Incremental and Weakly-Supervised Semantic Segmentation [39.025848280224785]
  We introduce a novel incremental class learning approach for semantic segmentation. Since each training step provides annotation only for a subset of all possible classes, pixels of the background class exhibit a semantic shift. We demonstrate the effectiveness of our approach with an extensive evaluation on the Pascal-VOC, ADE20K, and Cityscapes datasets.
  arXiv  Detail & Related papers  (2022-01-31T16:33:21Z)
• Few-Shot Object Detection via Association and DIscrimination [83.8472428718097]
  Few-shot object detection via Association and DIscrimination builds up a discriminative feature space for each novel class with two integral steps. Experiments on Pascal VOC and MS-COCO datasets demonstrate FADI achieves new SOTA performance, significantly improving the baseline in any shot/split by +18.7.
  arXiv  Detail & Related papers  (2021-11-23T05:04:06Z)
• Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations [18.655840060559168]
  This paper focuses on class incremental continual learning in semantic segmentation. New categories are made available over time while previous training data is not retained. The proposed continual learning scheme shapes the latent space to reduce forgetting whilst improving the recognition of novel classes.
  arXiv  Detail & Related papers  (2021-03-10T21:02:05Z)
• Prior Guided Feature Enrichment Network for Few-Shot Segmentation [64.91560451900125]
  State-of-the-art semantic segmentation methods require sufficient labeled data to achieve good results. Few-shot segmentation is proposed to tackle this problem by learning a model that quickly adapts to new classes with a few labeled support samples. Theses frameworks still face the challenge of generalization ability reduction on unseen classes due to inappropriate use of high-level semantic information.
  arXiv  Detail & Related papers  (2020-08-04T10:41:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.