Few-Shot Segmentation via Rich Prototype Generation and Recurrent
Prediction Enhancement
- URL: http://arxiv.org/abs/2210.00765v1
- Date: Mon, 3 Oct 2022 08:46:52 GMT
- Title: Few-Shot Segmentation via Rich Prototype Generation and Recurrent
Prediction Enhancement
- Authors: Hongsheng Wang, Xiaoqi Zhao, Youwei Pang, Jinqing Qi
- Abstract summary: We propose a rich prototype generation module (RPGM) and a recurrent prediction enhancement module (RPEM) to reinforce the prototype learning paradigm.
RPGM combines superpixel and K-means clustering to generate rich prototype features with complementary scale relationships.
RPEM utilizes the recurrent mechanism to design a round-way propagation decoder.
- Score: 12.614578133091168
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Prototype learning and decoder construction are the keys for few-shot
segmentation. However, existing methods use only a single prototype generation
mode, which can not cope with the intractable problem of objects with various
scales. Moreover, the one-way forward propagation adopted by previous methods
may cause information dilution from registered features during the decoding
process. In this research, we propose a rich prototype generation module (RPGM)
and a recurrent prediction enhancement module (RPEM) to reinforce the prototype
learning paradigm and build a unified memory-augmented decoder for few-shot
segmentation, respectively. Specifically, the RPGM combines superpixel and
K-means clustering to generate rich prototype features with complementary scale
relationships and adapt the scale gap between support and query images. The
RPEM utilizes the recurrent mechanism to design a round-way propagation
decoder. In this way, registered features can provide object-aware information
continuously. Experiments show that our method consistently outperforms other
competitors on two popular benchmarks PASCAL-${{5}^{i}}$ and COCO-${{20}^{i}}$.
Related papers
- Multi-Scale Grouped Prototypes for Interpretable Semantic Segmentation [7.372346036256517]
Prototypical part learning is emerging as a promising approach for making semantic segmentation interpretable.
We propose a method for interpretable semantic segmentation that leverages multi-scale image representation for prototypical part learning.
Experiments conducted on Pascal VOC, Cityscapes, and ADE20K demonstrate that the proposed method increases model sparsity, improves interpretability over existing prototype-based methods, and narrows the performance gap with the non-interpretable counterpart models.
arXiv Detail & Related papers (2024-09-14T17:52:59Z) - Query-guided Prototype Evolution Network for Few-Shot Segmentation [85.75516116674771]
We present a new method that integrates query features into the generation process of foreground and background prototypes.
Experimental results on the PASCAL-$5i$ and mirroring-$20i$ datasets attest to the substantial enhancements achieved by QPENet.
arXiv Detail & Related papers (2024-03-11T07:50:40Z) - Rethinking Few-shot 3D Point Cloud Semantic Segmentation [62.80639841429669]
This paper revisits few-shot 3D point cloud semantic segmentation (FS-PCS)
We focus on two significant issues in the state-of-the-art: foreground leakage and sparse point distribution.
To address these issues, we introduce a standardized FS-PCS setting, upon which a new benchmark is built.
arXiv Detail & Related papers (2024-03-01T15:14:47Z) - Multimodal Prototype-Enhanced Network for Few-Shot Action Recognition [40.329190454146996]
MultimOdal PRototype-ENhanced Network (MORN) uses semantic information of label texts as multimodal information to enhance prototypes.
We conduct extensive experiments on four popular few-shot action recognition datasets.
arXiv Detail & Related papers (2022-12-09T14:24:39Z) - Beyond the Prototype: Divide-and-conquer Proxies for Few-shot
Segmentation [63.910211095033596]
Few-shot segmentation aims to segment unseen-class objects given only a handful of densely labeled samples.
We propose a simple yet versatile framework in the spirit of divide-and-conquer.
Our proposed approach, named divide-and-conquer proxies (DCP), allows for the development of appropriate and reliable information.
arXiv Detail & Related papers (2022-04-21T06:21:14Z) - Rethinking Semantic Segmentation: A Prototype View [126.59244185849838]
We present a nonparametric semantic segmentation model based on non-learnable prototypes.
Our framework yields compelling results over several datasets.
We expect this work will provoke a rethink of the current de facto semantic segmentation model design.
arXiv Detail & Related papers (2022-03-28T21:15:32Z) - Contrastive Enhancement Using Latent Prototype for Few-Shot Segmentation [8.986743262828009]
Few-shot segmentation enables the model to recognize unseen classes with few annotated examples.
This paper proposes a contrastive enhancement approach using latent prototypes to leverage latent classes.
Our approach remarkably improves the performance of state-of-the-art methods for 1-shot and 5-shot segmentation.
arXiv Detail & Related papers (2022-03-08T14:02:32Z) - Dual Prototypical Contrastive Learning for Few-shot Semantic
Segmentation [55.339405417090084]
We propose a dual prototypical contrastive learning approach tailored to the few-shot semantic segmentation (FSS) task.
The main idea is to encourage the prototypes more discriminative by increasing inter-class distance while reducing intra-class distance in prototype feature space.
We demonstrate that the proposed dual contrastive learning approach outperforms state-of-the-art FSS methods on PASCAL-5i and COCO-20i datasets.
arXiv Detail & Related papers (2021-11-09T08:14:50Z) - Part-aware Prototype Network for Few-shot Semantic Segmentation [50.581647306020095]
We propose a novel few-shot semantic segmentation framework based on the prototype representation.
Our key idea is to decompose the holistic class representation into a set of part-aware prototypes.
We develop a novel graph neural network model to generate and enhance the proposed part-aware prototypes.
arXiv Detail & Related papers (2020-07-13T11:03:09Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.