Few-Shot Segmentation with Global and Local Contrastive Learning
- URL: http://arxiv.org/abs/2108.05293v1
- Date: Wed, 11 Aug 2021 15:52:22 GMT
- Title: Few-Shot Segmentation with Global and Local Contrastive Learning
- Authors: Weide Liu, Zhonghua Wu, Henghui Ding, Fayao Liu, Jie Lin, Guosheng Lin
- Abstract summary: We propose a prior extractor to learn the query information from the unlabeled images with our proposed global-local contrastive learning.
We generate the prior region maps for query images, which locate the objects, as guidance to perform cross interaction with support features.
Without bells and whistles, the proposed approach achieves new state-of-the-art performance for the few-shot segmentation task.
- Score: 51.677179037590356
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In this work, we address the challenging task of few-shot segmentation.
Previous few-shot segmentation methods mainly employ the information of support
images as guidance for query image segmentation. Although some works propose to
build cross-reference between support and query images, their extraction of
query information still depends on the support images. We here propose to
extract the information from the query itself independently to benefit the
few-shot segmentation task. To this end, we first propose a prior extractor to
learn the query information from the unlabeled images with our proposed
global-local contrastive learning. Then, we extract a set of predetermined
priors via this prior extractor. With the obtained priors, we generate the
prior region maps for query images, which locate the objects, as guidance to
perform cross interaction with support features. In such a way, the extraction
of query information is detached from the support branch, overcoming the
limitation by support, and could obtain more informative query clues to achieve
better interaction. Without bells and whistles, the proposed approach achieves
new state-of-the-art performance for the few-shot segmentation task on
PASCAL-5$^{i}$ and COCO datasets.
Related papers
- Segmentation-guided Attention for Visual Question Answering from Remote Sensing Images [1.6932802756478726]
Visual Question Answering for Remote Sensing (RSVQA) is a task that aims at answering natural language questions about the content of a remote sensing image.
We propose to embed an attention mechanism guided by segmentation into a RSVQA pipeline.
We provide a new VQA dataset that exploits very high-resolution RGB orthophotos annotated with 16 segmentation classes and question/answer pairs.
arXiv Detail & Related papers (2024-07-11T16:59:32Z) - MatchSeg: Towards Better Segmentation via Reference Image Matching [5.55078598520531]
Few-shot learning aims to overcome the need for annotated data by using a small labeled dataset, known as a support set, to guide predicting labels for new, unlabeled images.
Inspired by this paradigm, we introduce MatchSeg, a novel framework that enhances medical image segmentation through strategic reference image matching.
arXiv Detail & Related papers (2024-03-23T18:04:58Z) - Self-Correlation and Cross-Correlation Learning for Few-Shot Remote
Sensing Image Semantic Segmentation [27.59330408178435]
Few-shot remote sensing semantic segmentation aims at learning to segment target objects from a query image.
We propose a Self-Correlation and Cross-Correlation Learning Network for the few-shot remote sensing image semantic segmentation.
Our model enhances the generalization by considering both self-correlation and cross-correlation between support and query images.
arXiv Detail & Related papers (2023-09-11T21:53:34Z) - Few-shot Segmentation with Optimal Transport Matching and Message Flow [50.9853556696858]
It is essential for few-shot semantic segmentation to fully utilize the support information.
We propose a Correspondence Matching Network (CMNet) with an Optimal Transport Matching module.
Experiments on PASCAL VOC 2012, MS COCO, and FSS-1000 datasets show that our network achieves new state-of-the-art few-shot segmentation performance.
arXiv Detail & Related papers (2021-08-19T06:26:11Z) - Boosting Few-shot Semantic Segmentation with Transformers [81.43459055197435]
TRansformer-based Few-shot Semantic segmentation method (TRFS)
Our model consists of two modules: Global Enhancement Module (GEM) and Local Enhancement Module (LEM)
arXiv Detail & Related papers (2021-08-04T20:09:21Z) - Few-Shot Segmentation via Cycle-Consistent Transformer [74.49307213431952]
We focus on utilizing pixel-wise relationships between support and target images to facilitate the few-shot semantic segmentation task.
We propose using a novel cycle-consistent attention mechanism to filter out possible harmful support features.
Our proposed CyCTR leads to remarkable improvement compared to previous state-of-the-art methods.
arXiv Detail & Related papers (2021-06-04T07:57:48Z) - SCNet: Enhancing Few-Shot Semantic Segmentation by Self-Contrastive
Background Prototypes [56.387647750094466]
Few-shot semantic segmentation aims to segment novel-class objects in a query image with only a few annotated examples.
Most of advanced solutions exploit a metric learning framework that performs segmentation through matching each pixel to a learned foreground prototype.
This framework suffers from biased classification due to incomplete construction of sample pairs with the foreground prototype only.
arXiv Detail & Related papers (2021-04-19T11:21:47Z) - Self-Guided and Cross-Guided Learning for Few-Shot Segmentation [12.899804391102435]
We propose a self-guided learning approach for few-shot segmentation.
By making an initial prediction for the annotated support image, the covered and uncovered foreground regions are encoded to the primary and auxiliary support vectors.
By aggregating both primary and auxiliary support vectors, better segmentation performances are obtained on query images.
arXiv Detail & Related papers (2021-03-30T07:36:41Z) - CRNet: Cross-Reference Networks for Few-Shot Segmentation [59.85183776573642]
Few-shot segmentation aims to learn a segmentation model that can be generalized to novel classes with only a few training images.
With a cross-reference mechanism, our network can better find the co-occurrent objects in the two images.
Experiments on the PASCAL VOC 2012 dataset show that our network achieves state-of-the-art performance.
arXiv Detail & Related papers (2020-03-24T04:55:43Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.