Learning Meta-class Memory for Few-Shot Semantic Segmentation

• URL: http://arxiv.org/abs/2108.02958v2
• Date: Tue, 10 Aug 2021 12:23:10 GMT
• Title: Learning Meta-class Memory for Few-Shot Semantic Segmentation
• Authors: Zhonghua Wu, Xiangxi Shi, Guosheng lin, Jianfei Cai
• Abstract summary: We introduce the concept of meta-class, which is the meta information shareable among all classes. We propose a novel Meta-class Memory based few-shot segmentation method (MM-Net), where we introduce a set of learnable memory embeddings. Our proposed MM-Net achieves 37.5% mIoU on the COCO dataset in 1-shot setting, which is 5.1% higher than the previous state-of-the-art.
• Score: 90.28474742651422
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Currently, the state-of-the-art methods treat few-shot semantic segmentation task as a conditional foreground-background segmentation problem, assuming each class is independent. In this paper, we introduce the concept of meta-class, which is the meta information (e.g. certain middle-level features) shareable among all classes. To explicitly learn meta-class representations in few-shot segmentation task, we propose a novel Meta-class Memory based few-shot segmentation method (MM-Net), where we introduce a set of learnable memory embeddings to memorize the meta-class information during the base class training and transfer to novel classes during the inference stage. Moreover, for the $k$-shot scenario, we propose a novel image quality measurement module to select images from the set of support images. A high-quality class prototype could be obtained with the weighted sum of support image features based on the quality measure. Experiments on both PASCAL-$5^i$ and COCO dataset shows that our proposed method is able to achieve state-of-the-art results in both 1-shot and 5-shot settings. Particularly, our proposed MM-Net achieves 37.5\% mIoU on the COCO dataset in 1-shot setting, which is 5.1\% higher than the previous state-of-the-art.

Related papers

• Iterative Few-shot Semantic Segmentation from Image Label Text [36.53926941601841]
  Few-shot semantic segmentation aims to learn to segment unseen class objects with the guidance of only a few support images. We propose a general framework to generate coarse masks with the help of the powerful vision-language model CLIP. Our method owns an excellent generalization ability for the images in the wild and uncommon classes.
  arXiv  Detail & Related papers  (2023-03-10T01:48:14Z)
• A Joint Framework Towards Class-aware and Class-agnostic Alignment for Few-shot Segmentation [11.47479526463185]
  Few-shot segmentation aims to segment objects of unseen classes given only a few annotated support images. Most existing methods simply stitch query features with independent support prototypes and segment the query image by feeding the mixed features to a decoder. We propose a joint framework that combines more valuable class-aware and class-agnostic alignment guidance to facilitate the segmentation.
  arXiv  Detail & Related papers  (2022-11-02T17:33:25Z)
• MFNet: Multi-class Few-shot Segmentation Network with Pixel-wise Metric Learning [34.059257121606336]
  This work focuses on few-shot semantic segmentation, which is still a largely unexplored field. We first present a novel multi-way encoding and decoding architecture which effectively fuses multi-scale query information and multi-class support information into one query-support embedding. Experiments on standard benchmarks PASCAL-5i and COCO-20i show clear benefits of our method over the state of the art in few-shot segmentation.
  arXiv  Detail & Related papers  (2021-10-30T11:37:36Z)
• InfoSeg: Unsupervised Semantic Image Segmentation with Mutual Information Maximization [0.0]
  We propose a novel method for unsupervised image representation based on mutual information between local and global high-level image features. In the first step, we segment images based on local and global features. In the second step, we maximize the mutual information between local features and high-level features of their respective class.
  arXiv  Detail & Related papers  (2021-10-07T14:01:42Z)
• Boosting Few-shot Semantic Segmentation with Transformers [81.43459055197435]
  TRansformer-based Few-shot Semantic segmentation method (TRFS) Our model consists of two modules: Global Enhancement Module (GEM) and Local Enhancement Module (LEM)
  arXiv  Detail & Related papers  (2021-08-04T20:09:21Z)
• Rectifying the Shortcut Learning of Background: Shared Object Concentration for Few-Shot Image Recognition [101.59989523028264]
  Few-Shot image classification aims to utilize pretrained knowledge learned from a large-scale dataset to tackle a series of downstream classification tasks. We propose COSOC, a novel Few-Shot Learning framework, to automatically figure out foreground objects at both pretraining and evaluation stage.
  arXiv  Detail & Related papers  (2021-07-16T07:46:41Z)
• Few-Shot Segmentation via Cycle-Consistent Transformer [74.49307213431952]
  We focus on utilizing pixel-wise relationships between support and target images to facilitate the few-shot semantic segmentation task. We propose using a novel cycle-consistent attention mechanism to filter out possible harmful support features. Our proposed CyCTR leads to remarkable improvement compared to previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-06-04T07:57:48Z)
• Prior-Enhanced Few-Shot Segmentation with Meta-Prototypes [32.898636584823215]
  Few-shot segmentation(FSS) performance has been extensively promoted by introducing episodic training and class-wise prototypes. We propose the Prior-Enhanced network with Meta-Prototypes to tackle these limitations. Our method achieves the mean-IoU scores of 60.79% and 41.16% on PASCAL-$5i$ and COCO-$20i$, outperforming the state-of-the-art method by 3.49% and 5.64% in the 5-shot setting.
  arXiv  Detail & Related papers  (2021-06-01T15:34:30Z)
• SCNet: Enhancing Few-Shot Semantic Segmentation by Self-Contrastive Background Prototypes [56.387647750094466]
  Few-shot semantic segmentation aims to segment novel-class objects in a query image with only a few annotated examples. Most of advanced solutions exploit a metric learning framework that performs segmentation through matching each pixel to a learned foreground prototype. This framework suffers from biased classification due to incomplete construction of sample pairs with the foreground prototype only.
  arXiv  Detail & Related papers  (2021-04-19T11:21:47Z)
• Semantically Meaningful Class Prototype Learning for One-Shot Image Semantic Segmentation [58.96902899546075]
  One-shot semantic image segmentation aims to segment the object regions for the novel class with only one annotated image. Recent works adopt the episodic training strategy to mimic the expected situation at testing time. We propose to leverage the multi-class label information during the episodic training. It will encourage the network to generate more semantically meaningful features for each category.
  arXiv  Detail & Related papers  (2021-02-22T12:07:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.