Prototypical Kernel Learning and Open-set Foreground Perception for
Generalized Few-shot Semantic Segmentation
- URL: http://arxiv.org/abs/2308.04952v3
- Date: Sat, 19 Aug 2023 03:22:52 GMT
- Title: Prototypical Kernel Learning and Open-set Foreground Perception for
Generalized Few-shot Semantic Segmentation
- Authors: Kai Huang, Feigege Wang, Ye Xi, Yutao Gao
- Abstract summary: Generalized Few-shot Semantic (GFSS) extends Few-shot Semantic aggregation to segment unseen classes and seen classes during evaluation.
We address the aforementioned problems by jointing the prototypical kernel learning and open-set perception.
In addition, a foreground contextual perception module cooperating with conditional bias based inference is adopted to perform class-agnostic as well as open-set foreground detection.
- Score: 7.707161030443157
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Generalized Few-shot Semantic Segmentation (GFSS) extends Few-shot Semantic
Segmentation (FSS) to simultaneously segment unseen classes and seen classes
during evaluation. Previous works leverage additional branch or prototypical
aggregation to eliminate the constrained setting of FSS. However,
representation division and embedding prejudice, which heavily results in poor
performance of GFSS, have not been synthetical considered. We address the
aforementioned problems by jointing the prototypical kernel learning and
open-set foreground perception. Specifically, a group of learnable kernels is
proposed to perform segmentation with each kernel in charge of a stuff class.
Then, we explore to merge the prototypical learning to the update of base-class
kernels, which is consistent with the prototype knowledge aggregation of
few-shot novel classes. In addition, a foreground contextual perception module
cooperating with conditional bias based inference is adopted to perform
class-agnostic as well as open-set foreground detection, thus to mitigate the
embedding prejudice and prevent novel targets from being misclassified as
background. Moreover, we also adjust our method to the Class Incremental
Few-shot Semantic Segmentation (CIFSS) which takes the knowledge of novel
classes in a incremental stream. Extensive experiments on PASCAL-5i and
COCO-20i datasets demonstrate that our method performs better than previous
state-of-the-art.
Related papers
- Memory-guided Network with Uncertainty-based Feature Augmentation for Few-shot Semantic Segmentation [12.653336728447654]
We propose a class-shared memory (CSM) module consisting of a set of learnable memory vectors.
These memory vectors learn elemental object patterns from base classes during training whilst re-encoding query features during both training and inference.
We integrate CSM and UFA into representative FSS works, with experimental results on the widely-used PASCAL-5$i$ and COCO-20$i$ datasets.
arXiv Detail & Related papers (2024-06-01T19:53:25Z) - Rethinking Few-shot 3D Point Cloud Semantic Segmentation [62.80639841429669]
This paper revisits few-shot 3D point cloud semantic segmentation (FS-PCS)
We focus on two significant issues in the state-of-the-art: foreground leakage and sparse point distribution.
To address these issues, we introduce a standardized FS-PCS setting, upon which a new benchmark is built.
arXiv Detail & Related papers (2024-03-01T15:14:47Z) - Few-Shot Learning for Annotation-Efficient Nucleus Instance Segmentation [50.407071700154674]
We propose to formulate annotation-efficient nucleus instance segmentation from the perspective of few-shot learning (FSL)
Our work was motivated by that, with the prosperity of computational pathology, an increasing number of fully-annotated datasets are publicly accessible.
Extensive experiments on a couple of publicly accessible datasets demonstrate that SGFSIS can outperform other annotation-efficient learning baselines.
arXiv Detail & Related papers (2024-02-26T03:49:18Z) - Background Clustering Pre-training for Few-shot Segmentation [11.954463256405967]
Recent few-shot segmentation (FSS) methods introduce an extra pre-training stage before meta-training to obtain a stronger backbone.
Current pre-training scheme suffers from the merged background problem: only base classes are labelled as foregrounds.
We propose a new pre-training scheme for FSS via decoupling the novel classes from background, called Background Clustering Pre-Training (BCPT)
arXiv Detail & Related papers (2023-12-06T07:16:32Z) - Harmonizing Base and Novel Classes: A Class-Contrastive Approach for
Generalized Few-Shot Segmentation [78.74340676536441]
We propose a class contrastive loss and a class relationship loss to regulate prototype updates and encourage a large distance between prototypes.
Our proposed approach achieves new state-of-the-art performance for the generalized few-shot segmentation task on PASCAL VOC and MS COCO datasets.
arXiv Detail & Related papers (2023-03-24T00:30:25Z) - Prediction Calibration for Generalized Few-shot Semantic Segmentation [101.69940565204816]
Generalized Few-shot Semantic (GFSS) aims to segment each image pixel into either base classes with abundant training examples or novel classes with only a handful of (e.g., 1-5) training images per class.
We build a cross-attention module that guides the classifier's final prediction using the fused multi-level features.
Our PCN outperforms the state-the-art alternatives by large margins.
arXiv Detail & Related papers (2022-10-15T13:30:12Z) - Learning What Not to Segment: A New Perspective on Few-Shot Segmentation [63.910211095033596]
Recently few-shot segmentation (FSS) has been extensively developed.
This paper proposes a fresh and straightforward insight to alleviate the problem.
In light of the unique nature of the proposed approach, we also extend it to a more realistic but challenging setting.
arXiv Detail & Related papers (2022-03-15T03:08:27Z) - Generalized Few-shot Semantic Segmentation [68.69434831359669]
We introduce a new benchmark called Generalized Few-Shot Semantic (GFS-Seg) to analyze the ability of simultaneously segmenting the novel categories.
It is the first study showing that previous representative state-of-the-art generalizations fall short in GFS-Seg.
We propose the Context-Aware Prototype Learning (CAPL) that significantly improves performance by 1) leveraging the co-occurrence prior knowledge from support samples, and 2) dynamically enriching contextual information to the conditioned, on the content of each query image.
arXiv Detail & Related papers (2020-10-11T10:13:21Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.