Query-guided Prototype Evolution Network for Few-Shot Segmentation

• URL: http://arxiv.org/abs/2403.06488v1
• Date: Mon, 11 Mar 2024 07:50:40 GMT
• Title: Query-guided Prototype Evolution Network for Few-Shot Segmentation
• Authors: Runmin Cong, Hang Xiong, Jinpeng Chen, Wei Zhang, Qingming Huang, and Yao Zhao
• Abstract summary: We present a new method that integrates query features into the generation process of foreground and background prototypes. Experimental results on the PASCAL-$5i$ and mirroring-$20i$ datasets attest to the substantial enhancements achieved by QPENet.
• Score: 85.75516116674771
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Previous Few-Shot Segmentation (FSS) approaches exclusively utilize support features for prototype generation, neglecting the specific requirements of the query. To address this, we present the Query-guided Prototype Evolution Network (QPENet), a new method that integrates query features into the generation process of foreground and background prototypes, thereby yielding customized prototypes attuned to specific queries. The evolution of the foreground prototype is accomplished through a \textit{support-query-support} iterative process involving two new modules: Pseudo-prototype Generation (PPG) and Dual Prototype Evolution (DPE). The PPG module employs support features to create an initial prototype for the preliminary segmentation of the query image, resulting in a pseudo-prototype reflecting the unique needs of the current query. Subsequently, the DPE module performs reverse segmentation on support images using this pseudo-prototype, leading to the generation of evolved prototypes, which can be considered as custom solutions. As for the background prototype, the evolution begins with a global background prototype that represents the generalized features of all training images. We also design a Global Background Cleansing (GBC) module to eliminate potential adverse components mirroring the characteristics of the current foreground class. Experimental results on the PASCAL-$5^i$ and COCO-$20^i$ datasets attest to the substantial enhancements achieved by QPENet over prevailing state-of-the-art techniques, underscoring the validity of our ideas.

Related papers

• Symmetrical Joint Learning Support-query Prototypes for Few-shot Segmentation [33.33249452130038]
  We propose Sym-Net, a novel framework for Few-Shot (FSS) that addresses the critical issue of intra-class variation. We jointly learn both query and support prototypes in a symmetrical manner, ensuring that the learning process does not favor one set (support or query) over the other. Experimental results show that the proposed Sym-Net outperforms state-of-the-art models.
  arXiv  Detail & Related papers  (2024-07-27T17:37:56Z)
• Intermediate Prototype Mining Transformer for Few-Shot Semantic Segmentation [119.51445225693382]
  Few-shot semantic segmentation aims to segment the target objects in query under the condition of a few annotated support images. We introduce an intermediate prototype for mining both deterministic category information from the support and adaptive category knowledge from the query. In each IPMT layer, we propagate the object information in both support and query features to the prototype and then use it to activate the query feature map.
  arXiv  Detail & Related papers  (2022-10-13T06:45:07Z)
• Few-Shot Segmentation via Rich Prototype Generation and Recurrent Prediction Enhancement [12.614578133091168]
  We propose a rich prototype generation module (RPGM) and a recurrent prediction enhancement module (RPEM) to reinforce the prototype learning paradigm. RPGM combines superpixel and K-means clustering to generate rich prototype features with complementary scale relationships. RPEM utilizes the recurrent mechanism to design a round-way propagation decoder.
  arXiv  Detail & Related papers  (2022-10-03T08:46:52Z)
• ProtoPFormer: Concentrating on Prototypical Parts in Vision Transformers for Interpretable Image Recognition [32.34322644235324]
  Prototypical part network (ProtoPNet) has drawn wide attention and boosted many follow-up studies due to its self-explanatory property for explainable artificial intelligence (XAI) When directly applying ProtoPNet on vision transformer (ViT) backbones, learned prototypes have a relatively high probability of being activated by the background and pay less attention to the foreground. This paper proposes prototypical part transformer (ProtoPFormer) for appropriately and effectively applying the prototype-based method with ViTs for interpretable image recognition.
  arXiv  Detail & Related papers  (2022-08-22T16:36:32Z)
• Rethinking Semantic Segmentation: A Prototype View [126.59244185849838]
  We present a nonparametric semantic segmentation model based on non-learnable prototypes. Our framework yields compelling results over several datasets. We expect this work will provoke a rethink of the current de facto semantic segmentation model design.
  arXiv  Detail & Related papers  (2022-03-28T21:15:32Z)
• Dual Prototypical Contrastive Learning for Few-shot Semantic Segmentation [55.339405417090084]
  We propose a dual prototypical contrastive learning approach tailored to the few-shot semantic segmentation (FSS) task. The main idea is to encourage the prototypes more discriminative by increasing inter-class distance while reducing intra-class distance in prototype feature space. We demonstrate that the proposed dual contrastive learning approach outperforms state-of-the-art FSS methods on PASCAL-5i and COCO-20i datasets.
  arXiv  Detail & Related papers  (2021-11-09T08:14:50Z)
• Prototype Completion for Few-Shot Learning [13.63424509914303]
  Few-shot learning aims to recognize novel classes with few examples. Pre-training based methods effectively tackle the problem by pre-training a feature extractor and then fine-tuning it through the nearest centroid based meta-learning. We propose a novel prototype completion based meta-learning framework.
  arXiv  Detail & Related papers  (2021-08-11T03:44:00Z)
• SCNet: Enhancing Few-Shot Semantic Segmentation by Self-Contrastive Background Prototypes [56.387647750094466]
  Few-shot semantic segmentation aims to segment novel-class objects in a query image with only a few annotated examples. Most of advanced solutions exploit a metric learning framework that performs segmentation through matching each pixel to a learned foreground prototype. This framework suffers from biased classification due to incomplete construction of sample pairs with the foreground prototype only.
  arXiv  Detail & Related papers  (2021-04-19T11:21:47Z)
• Learning Sparse Prototypes for Text Generation [120.38555855991562]
  Prototype-driven text generation is inefficient at test time as a result of needing to store and index the entire training corpus. We propose a novel generative model that automatically learns a sparse prototype support set that achieves strong language modeling performance. In experiments, our model outperforms previous prototype-driven language models while achieving up to a 1000x memory reduction.
  arXiv  Detail & Related papers  (2020-06-29T19:41:26Z)
• Prototype Refinement Network for Few-Shot Segmentation [6.777019450570474]
  We propose a Prototype Refinement Network (PRNet) to attack the challenge of few-shot segmentation. It firstly learns to bidirectionally extract prototypes from both support and query images of the known classes. PRNet significantly outperforms existing methods by a large margin of 13.1% in 1-shot setting.
  arXiv  Detail & Related papers  (2020-02-10T07:06:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.