Background Clustering Pre-training for Few-shot Segmentation
- URL: http://arxiv.org/abs/2312.03322v1
- Date: Wed, 6 Dec 2023 07:16:32 GMT
- Title: Background Clustering Pre-training for Few-shot Segmentation
- Authors: Zhimiao Yu, Tiancheng Lin, Yi Xu
- Abstract summary: Recent few-shot segmentation (FSS) methods introduce an extra pre-training stage before meta-training to obtain a stronger backbone.
Current pre-training scheme suffers from the merged background problem: only base classes are labelled as foregrounds.
We propose a new pre-training scheme for FSS via decoupling the novel classes from background, called Background Clustering Pre-Training (BCPT)
- Score: 11.954463256405967
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Recent few-shot segmentation (FSS) methods introduce an extra pre-training
stage before meta-training to obtain a stronger backbone, which has become a
standard step in few-shot learning. Despite the effectiveness, current
pre-training scheme suffers from the merged background problem: only base
classes are labelled as foregrounds, making it hard to distinguish between
novel classes and actual background. In this paper, we propose a new
pre-training scheme for FSS via decoupling the novel classes from background,
called Background Clustering Pre-Training (BCPT). Specifically, we adopt online
clustering to the pixel embeddings of merged background to explore the
underlying semantic structures, bridging the gap between pre-training and
adaptation to novel classes. Given the clustering results, we further propose
the background mining loss and leverage base classes to guide the clustering
process, improving the quality and stability of clustering results. Experiments
on PASCAL-5i and COCO-20i show that BCPT yields advanced performance. Code will
be available.
Related papers
- SLCA++: Unleash the Power of Sequential Fine-tuning for Continual Learning with Pre-training [68.7896349660824]
We present an in-depth analysis of the progressive overfitting problem from the lens of Seq FT.
Considering that the overly fast representation learning and the biased classification layer constitute this particular problem, we introduce the advanced Slow Learner with Alignment (S++) framework.
Our approach involves a Slow Learner to selectively reduce the learning rate of backbone parameters, and a Alignment to align the disjoint classification layers in a post-hoc fashion.
arXiv Detail & Related papers (2024-08-15T17:50:07Z) - Embedding And Clustering Your Data Can Improve Contrastive Pretraining [0.0]
We explore extending training data stratification beyond source granularity by leveraging a pretrained text embedding model and the classic k-means clustering algorithm.
Experimentally, we observe a notable increase in NDCG@10 when pretraining a BERT-based text embedding model on query-passage pairs from the MSMARCO passage retrieval dataset.
arXiv Detail & Related papers (2024-07-26T17:36:40Z) - Class Similarity Transition: Decoupling Class Similarities and Imbalance from Generalized Few-shot Segmentation [17.33292771556997]
This paper focuses on the relevance between base and novel classes, and improves Generalized Few-shot (GFSS)
We first propose a similarity transition matrix to guide the learning of novel classes with base class knowledge.
We then leverage the Label-Distribution-Aware Margin (LDAM) loss and Transductive Inference to the GFSS task to address the problem of class imbalance.
arXiv Detail & Related papers (2024-04-08T00:13:05Z) - Rethinking Few-shot 3D Point Cloud Semantic Segmentation [62.80639841429669]
This paper revisits few-shot 3D point cloud semantic segmentation (FS-PCS)
We focus on two significant issues in the state-of-the-art: foreground leakage and sparse point distribution.
To address these issues, we introduce a standardized FS-PCS setting, upon which a new benchmark is built.
arXiv Detail & Related papers (2024-03-01T15:14:47Z) - Read Between the Layers: Leveraging Multi-Layer Representations for Rehearsal-Free Continual Learning with Pre-Trained Models [15.847302755988506]
We address the Continual Learning problem, wherein a model must learn a sequence of tasks from non-stationary distributions.
We propose LayUP, a new prototype-based approach to CL that leverages second-order feature statistics from multiple intermediate layers of a pre-trained network.
Our results demonstrate that fully exhausting the representational capacities of pre-trained models in CL goes well beyond their final embeddings.
arXiv Detail & Related papers (2023-12-13T13:11:44Z) - Prototypical Kernel Learning and Open-set Foreground Perception for
Generalized Few-shot Semantic Segmentation [7.707161030443157]
Generalized Few-shot Semantic (GFSS) extends Few-shot Semantic aggregation to segment unseen classes and seen classes during evaluation.
We address the aforementioned problems by jointing the prototypical kernel learning and open-set perception.
In addition, a foreground contextual perception module cooperating with conditional bias based inference is adopted to perform class-agnostic as well as open-set foreground detection.
arXiv Detail & Related papers (2023-08-09T13:38:52Z) - RanPAC: Random Projections and Pre-trained Models for Continual Learning [59.07316955610658]
Continual learning (CL) aims to learn different tasks (such as classification) in a non-stationary data stream without forgetting old ones.
We propose a concise and effective approach for CL with pre-trained models.
arXiv Detail & Related papers (2023-07-05T12:49:02Z) - Harmonizing Base and Novel Classes: A Class-Contrastive Approach for
Generalized Few-Shot Segmentation [78.74340676536441]
We propose a class contrastive loss and a class relationship loss to regulate prototype updates and encourage a large distance between prototypes.
Our proposed approach achieves new state-of-the-art performance for the generalized few-shot segmentation task on PASCAL VOC and MS COCO datasets.
arXiv Detail & Related papers (2023-03-24T00:30:25Z) - Mining Unseen Classes via Regional Objectness: A Simple Baseline for
Incremental Segmentation [57.80416375466496]
Incremental or continual learning has been extensively studied for image classification tasks to alleviate catastrophic forgetting.
We propose a simple yet effective method in this paper, named unseen Classes via Regional Objectness for Mining (MicroSeg)
Our MicroSeg is based on the assumption that background regions with strong objectness possibly belong to those concepts in the historical or future stages.
In this way, the distribution characterizes of old concepts in the feature space could be better perceived, relieving the catastrophic forgetting caused by the background shift accordingly.
arXiv Detail & Related papers (2022-11-13T10:06:17Z) - Novel Class Discovery in Semantic Segmentation [104.30729847367104]
We introduce a new setting of Novel Class Discovery in Semantic (NCDSS)
It aims at segmenting unlabeled images containing new classes given prior knowledge from a labeled set of disjoint classes.
In NCDSS, we need to distinguish the objects and background, and to handle the existence of multiple classes within an image.
We propose the Entropy-based Uncertainty Modeling and Self-training (EUMS) framework to overcome noisy pseudo-labels.
arXiv Detail & Related papers (2021-12-03T13:31:59Z) - Tackling Catastrophic Forgetting and Background Shift in Continual
Semantic Segmentation [35.2461834832935]
Continual learning for semantic segmentation (CSS) is an emerging trend that consists in updating an old model by sequentially adding new classes.
In this paper, we propose Local POD, a multi-scale pooling distillation scheme that preserves long- and short-range spatial relationships.
We also introduce a novel rehearsal method that is particularly suited for segmentation.
arXiv Detail & Related papers (2021-06-29T11:57:21Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.