Leveraging GAN Priors for Few-Shot Part Segmentation
- URL: http://arxiv.org/abs/2207.13428v1
- Date: Wed, 27 Jul 2022 10:17:07 GMT
- Title: Leveraging GAN Priors for Few-Shot Part Segmentation
- Authors: Mengya Han, Heliang Zheng, Chaoyue Wang, Yong Luo, Han Hu, Bo Du
- Abstract summary: Few-shot part segmentation aims to separate different parts of an object given only a few samples.
We propose to learn task-specific features in a "pre-training"-"fine-tuning" paradigm.
- Score: 43.35150430895919
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Few-shot part segmentation aims to separate different parts of an object
given only a few annotated samples. Due to the challenge of limited data,
existing works mainly focus on learning classifiers over pre-trained features,
failing to learn task-specific features for part segmentation. In this paper,
we propose to learn task-specific features in a "pre-training"-"fine-tuning"
paradigm. We conduct prompt designing to reduce the gap between the pre-train
task (i.e., image generation) and the downstream task (i.e., part
segmentation), so that the GAN priors for generation can be leveraged for
segmentation. This is achieved by projecting part segmentation maps into the
RGB space and conducting interpolation between RGB segmentation maps and
original images. Specifically, we design a fine-tuning strategy to
progressively tune an image generator into a segmentation generator, where the
supervision of the generator varying from images to segmentation maps by
interpolation. Moreover, we propose a two-stream architecture, i.e., a
segmentation stream to generate task-specific features, and an image stream to
provide spatial constraints. The image stream can be regarded as a
self-supervised auto-encoder, and this enables our model to benefit from
large-scale support images. Overall, this work is an attempt to explore the
internal relevance between generation tasks and perception tasks by prompt
designing. Extensive experiments show that our model can achieve
state-of-the-art performance on several part segmentation datasets.
Related papers
- Framework-agnostic Semantically-aware Global Reasoning for Segmentation [29.69187816377079]
We propose a component that learns to project image features into latent representations and reason between them.
Our design encourages the latent regions to represent semantic concepts by ensuring that the activated regions are spatially disjoint.
Our latent tokens are semantically interpretable and diverse and provide a rich set of features that can be transferred to downstream tasks.
arXiv Detail & Related papers (2022-12-06T21:42:05Z) - Progressively Dual Prior Guided Few-shot Semantic Segmentation [57.37506990980975]
Few-shot semantic segmentation task aims at performing segmentation in query images with a few annotated support samples.
We propose a progressively dual prior guided few-shot semantic segmentation network.
arXiv Detail & Related papers (2022-11-20T16:19:47Z) - EAA-Net: Rethinking the Autoencoder Architecture with Intra-class
Features for Medical Image Segmentation [4.777011444412729]
We propose a light-weight end-to-end segmentation framework based on multi-task learning, termed Edge Attention autoencoder Network (EAA-Net)
Our approach not only utilizes the segmentation network to obtain inter-class features, but also applies the reconstruction network to extract intra-class features among the foregrounds.
Experimental results show that our method performs well in medical image segmentation tasks.
arXiv Detail & Related papers (2022-08-19T07:42:55Z) - Instance Segmentation of Unlabeled Modalities via Cyclic Segmentation
GAN [27.936725483892076]
We propose a novel Cyclic Generative Adrial Network (CySGAN) that conducts image translation and instance segmentation jointly.
We benchmark our approach on the task of 3D neuronal nuclei segmentation with annotated electron microscopy (EM) images and unlabeled expansion microscopy (ExM) data.
arXiv Detail & Related papers (2022-04-06T20:46:39Z) - Learning with Free Object Segments for Long-Tailed Instance Segmentation [15.563842274862314]
We find that an abundance of instance segments can potentially be obtained freely from object-centric im-ages.
Motivated by these insights, we propose FreeSeg for extracting and leveraging these "free" object segments.
FreeSeg achieves state-of-the-art accuracy for segmenting rare object categories.
arXiv Detail & Related papers (2022-02-22T19:06:16Z) - Prototypical Cross-Attention Networks for Multiple Object Tracking and
Segmentation [95.74244714914052]
Multiple object tracking and segmentation requires detecting, tracking, and segmenting objects belonging to a set of given classes.
We propose Prototypical Cross-Attention Network (PCAN), capable of leveraging rich-temporal information online.
PCAN outperforms current video instance tracking and segmentation competition winners on Youtube-VIS and BDD100K datasets.
arXiv Detail & Related papers (2021-06-22T17:57:24Z) - SCNet: Enhancing Few-Shot Semantic Segmentation by Self-Contrastive
Background Prototypes [56.387647750094466]
Few-shot semantic segmentation aims to segment novel-class objects in a query image with only a few annotated examples.
Most of advanced solutions exploit a metric learning framework that performs segmentation through matching each pixel to a learned foreground prototype.
This framework suffers from biased classification due to incomplete construction of sample pairs with the foreground prototype only.
arXiv Detail & Related papers (2021-04-19T11:21:47Z) - Part-aware Prototype Network for Few-shot Semantic Segmentation [50.581647306020095]
We propose a novel few-shot semantic segmentation framework based on the prototype representation.
Our key idea is to decompose the holistic class representation into a set of part-aware prototypes.
We develop a novel graph neural network model to generate and enhance the proposed part-aware prototypes.
arXiv Detail & Related papers (2020-07-13T11:03:09Z) - CRNet: Cross-Reference Networks for Few-Shot Segmentation [59.85183776573642]
Few-shot segmentation aims to learn a segmentation model that can be generalized to novel classes with only a few training images.
With a cross-reference mechanism, our network can better find the co-occurrent objects in the two images.
Experiments on the PASCAL VOC 2012 dataset show that our network achieves state-of-the-art performance.
arXiv Detail & Related papers (2020-03-24T04:55:43Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.