Self-Support Few-Shot Semantic Segmentation

• URL: http://arxiv.org/abs/2207.11549v1
• Date: Sat, 23 Jul 2022 16:28:07 GMT
• Title: Self-Support Few-Shot Semantic Segmentation
• Authors: Qi Fan, Wenjie Pei, Yu-Wing Tai, Chi-Keung Tang
• Abstract summary: We propose a novel self-support matching strategy, which uses query prototypes to match query features. We also propose an adaptive self-support background prototype generation module and self-support loss to further facilitate the self-support matching procedure. Our self-support network substantially improves the prototype quality, benefits more improvement from stronger backbones and more supports, and achieves SOTA on multiple datasets.
• Score: 72.43667576285445
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing few-shot segmentation methods have achieved great progress based on the support-query matching framework. But they still heavily suffer from the limited coverage of intra-class variations from the few-shot supports provided. Motivated by the simple Gestalt principle that pixels belonging to the same object are more similar than those to different objects of same class, we propose a novel self-support matching strategy to alleviate this problem, which uses query prototypes to match query features, where the query prototypes are collected from high-confidence query predictions. This strategy can effectively capture the consistent underlying characteristics of the query objects, and thus fittingly match query features. We also propose an adaptive self-support background prototype generation module and self-support loss to further facilitate the self-support matching procedure. Our self-support network substantially improves the prototype quality, benefits more improvement from stronger backbones and more supports, and achieves SOTA on multiple datasets. Codes are at \url{https://github.com/fanq15/SSP}.

Related papers

• Symmetrical Joint Learning Support-query Prototypes for Few-shot Segmentation [33.33249452130038]
  We propose Sym-Net, a novel framework for Few-Shot (FSS) that addresses the critical issue of intra-class variation. We jointly learn both query and support prototypes in a symmetrical manner, ensuring that the learning process does not favor one set (support or query) over the other. Experimental results show that the proposed Sym-Net outperforms state-of-the-art models.
  arXiv  Detail & Related papers  (2024-07-27T17:37:56Z)
• A Refreshed Similarity-based Upsampler for Direct High-Ratio Feature Upsampling [54.05517338122698]
  We propose an explicitly controllable query-key feature alignment from both semantic-aware and detail-aware perspectives. We also develop a fine-grained neighbor selection strategy on HR features, which is simple yet effective for alleviating mosaic artifacts. Our proposed ReSFU framework consistently achieves satisfactory performance on different segmentation applications.
  arXiv  Detail & Related papers  (2024-07-02T14:12:21Z)
• Prototype as Query for Few Shot Semantic Segmentation [7.380266341356485]
  Few-shot Semantic (FSS) was proposed to segment unseen classes in a query image, referring to only a few examples named support images. We propose a framework built upon Transformer termed as ProtoFormer to fully capture spatial details in query features.
  arXiv  Detail & Related papers  (2022-11-27T08:41:50Z)
• CAD: Co-Adapting Discriminative Features for Improved Few-Shot Classification [11.894289991529496]
  Few-shot classification is a challenging problem that aims to learn a model that can adapt to unseen classes given a few labeled samples. Recent approaches pre-train a feature extractor, and then fine-tune for episodic meta-learning. We propose a strategy to cross-attend and re-weight discriminative features for few-shot classification.
  arXiv  Detail & Related papers  (2022-03-25T06:14:51Z)
• Contrastive Enhancement Using Latent Prototype for Few-Shot Segmentation [8.986743262828009]
  Few-shot segmentation enables the model to recognize unseen classes with few annotated examples. This paper proposes a contrastive enhancement approach using latent prototypes to leverage latent classes. Our approach remarkably improves the performance of state-of-the-art methods for 1-shot and 5-shot segmentation.
  arXiv  Detail & Related papers  (2022-03-08T14:02:32Z)
• APANet: Adaptive Prototypes Alignment Network for Few-Shot Semantic Segmentation [56.387647750094466]
  Few-shot semantic segmentation aims to segment novel-class objects in a given query image with only a few labeled support images. Most advanced solutions exploit a metric learning framework that performs segmentation through matching each query feature to a learned class-specific prototype. We present an adaptive prototype representation by introducing class-specific and class-agnostic prototypes.
  arXiv  Detail & Related papers  (2021-11-24T04:38:37Z)
• Contextualizing Meta-Learning via Learning to Decompose [125.76658595408607]
  We propose Learning to Decompose Network (LeadNet) to contextualize the meta-learned support-to-target'' strategy. LeadNet learns to automatically select the strategy associated with the right via incorporating the change of comparison across contexts with polysemous embeddings.
  arXiv  Detail & Related papers  (2021-06-15T13:10:56Z)
• Few-Shot Segmentation via Cycle-Consistent Transformer [74.49307213431952]
  We focus on utilizing pixel-wise relationships between support and target images to facilitate the few-shot semantic segmentation task. We propose using a novel cycle-consistent attention mechanism to filter out possible harmful support features. Our proposed CyCTR leads to remarkable improvement compared to previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-06-04T07:57:48Z)
• Prior Guided Feature Enrichment Network for Few-Shot Segmentation [64.91560451900125]
  State-of-the-art semantic segmentation methods require sufficient labeled data to achieve good results. Few-shot segmentation is proposed to tackle this problem by learning a model that quickly adapts to new classes with a few labeled support samples. Theses frameworks still face the challenge of generalization ability reduction on unseen classes due to inappropriate use of high-level semantic information.
  arXiv  Detail & Related papers  (2020-08-04T10:41:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.