A Joint Framework Towards Class-aware and Class-agnostic Alignment for Few-shot Segmentation

• URL: http://arxiv.org/abs/2211.01310v1
• Date: Wed, 2 Nov 2022 17:33:25 GMT
• Title: A Joint Framework Towards Class-aware and Class-agnostic Alignment for Few-shot Segmentation
• Authors: Kai Huang and Mingfei Cheng and Yang Wang and Bochen Wang and Ye Xi and Feigege Wang and Peng Chen
• Abstract summary: Few-shot segmentation aims to segment objects of unseen classes given only a few annotated support images. Most existing methods simply stitch query features with independent support prototypes and segment the query image by feeding the mixed features to a decoder. We propose a joint framework that combines more valuable class-aware and class-agnostic alignment guidance to facilitate the segmentation.
• Score: 11.47479526463185
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Few-shot segmentation (FSS) aims to segment objects of unseen classes given only a few annotated support images. Most existing methods simply stitch query features with independent support prototypes and segment the query image by feeding the mixed features to a decoder. Although significant improvements have been achieved, existing methods are still face class biases due to class variants and background confusion. In this paper, we propose a joint framework that combines more valuable class-aware and class-agnostic alignment guidance to facilitate the segmentation. Specifically, we design a hybrid alignment module which establishes multi-scale query-support correspondences to mine the most relevant class-aware information for each query image from the corresponding support features. In addition, we explore utilizing base-classes knowledge to generate class-agnostic prior mask which makes a distinction between real background and foreground by highlighting all object regions, especially those of unseen classes. By jointly aggregating class-aware and class-agnostic alignment guidance, better segmentation performances are obtained on query images. Extensive experiments on PASCAL-$5^i$ and COCO-$20^i$ datasets demonstrate that our proposed joint framework performs better, especially on the 1-shot setting.

Related papers

• Boosting Semantic Segmentation from the Perspective of Explicit Class Embeddings [19.997929884477628]
  We explore the mechanism of class embeddings and have an insight that more explicit and meaningful class embeddings can be generated based on class masks purposely. We propose ECENet, a new segmentation paradigm, in which class embeddings are obtained and enhanced explicitly during interacting with multi-stage image features. Our ECENet outperforms its counterparts on the ADE20K dataset with much less computational cost and achieves new state-of-the-art results on PASCAL-Context dataset.
  arXiv  Detail & Related papers  (2023-08-24T16:16:10Z)
• Masked Cross-image Encoding for Few-shot Segmentation [16.445813548503708]
  Few-shot segmentation (FSS) is a dense prediction task that aims to infer the pixel-wise labels of unseen classes using only a limited number of annotated images. We propose a joint learning method termed Masked Cross-Image MCE, which is designed to capture common visual properties that describe object details and to learn bidirectional inter-image dependencies that enhance feature interaction.
  arXiv  Detail & Related papers  (2023-08-22T05:36:39Z)
• APANet: Adaptive Prototypes Alignment Network for Few-Shot Semantic Segmentation [56.387647750094466]
  Few-shot semantic segmentation aims to segment novel-class objects in a given query image with only a few labeled support images. Most advanced solutions exploit a metric learning framework that performs segmentation through matching each query feature to a learned class-specific prototype. We present an adaptive prototype representation by introducing class-specific and class-agnostic prototypes.
  arXiv  Detail & Related papers  (2021-11-24T04:38:37Z)
• MFNet: Multi-class Few-shot Segmentation Network with Pixel-wise Metric Learning [34.059257121606336]
  This work focuses on few-shot semantic segmentation, which is still a largely unexplored field. We first present a novel multi-way encoding and decoding architecture which effectively fuses multi-scale query information and multi-class support information into one query-support embedding. Experiments on standard benchmarks PASCAL-5i and COCO-20i show clear benefits of our method over the state of the art in few-shot segmentation.
  arXiv  Detail & Related papers  (2021-10-30T11:37:36Z)
• Learning Meta-class Memory for Few-Shot Semantic Segmentation [90.28474742651422]
  We introduce the concept of meta-class, which is the meta information shareable among all classes. We propose a novel Meta-class Memory based few-shot segmentation method (MM-Net), where we introduce a set of learnable memory embeddings. Our proposed MM-Net achieves 37.5% mIoU on the COCO dataset in 1-shot setting, which is 5.1% higher than the previous state-of-the-art.
  arXiv  Detail & Related papers  (2021-08-06T06:29:59Z)
• SCNet: Enhancing Few-Shot Semantic Segmentation by Self-Contrastive Background Prototypes [56.387647750094466]
  Few-shot semantic segmentation aims to segment novel-class objects in a query image with only a few annotated examples. Most of advanced solutions exploit a metric learning framework that performs segmentation through matching each pixel to a learned foreground prototype. This framework suffers from biased classification due to incomplete construction of sample pairs with the foreground prototype only.
  arXiv  Detail & Related papers  (2021-04-19T11:21:47Z)
• Semantically Meaningful Class Prototype Learning for One-Shot Image Semantic Segmentation [58.96902899546075]
  One-shot semantic image segmentation aims to segment the object regions for the novel class with only one annotated image. Recent works adopt the episodic training strategy to mimic the expected situation at testing time. We propose to leverage the multi-class label information during the episodic training. It will encourage the network to generate more semantically meaningful features for each category.
  arXiv  Detail & Related papers  (2021-02-22T12:07:35Z)
• BriNet: Towards Bridging the Intra-class and Inter-class Gaps in One-Shot Segmentation [84.2925550033094]
  Few-shot segmentation focuses on the generalization of models to segment unseen object instances with limited training samples. We propose a framework, BriNet, to bridge the gaps between the extracted features of the query and support images. The effectiveness of our framework is demonstrated by experimental results, which outperforms other competitive methods.
  arXiv  Detail & Related papers  (2020-08-14T07:45:50Z)
• A Few-Shot Sequential Approach for Object Counting [63.82757025821265]
  We introduce a class attention mechanism that sequentially attends to objects in the image and extracts their relevant features. The proposed technique is trained on point-level annotations and uses a novel loss function that disentangles class-dependent and class-agnostic aspects of the model. We present our results on a variety of object-counting/detection datasets, including FSOD and MS COCO.
  arXiv  Detail & Related papers  (2020-07-03T18:23:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.