Query Semantic Reconstruction for Background in Few-Shot Segmentation

• URL: http://arxiv.org/abs/2210.12055v1
• Date: Fri, 21 Oct 2022 15:49:16 GMT
• Title: Query Semantic Reconstruction for Background in Few-Shot Segmentation
• Authors: Haoyan Guan, Michael Spratling
• Abstract summary: Few-shot segmentation (FSS) aims to segment unseen classes using a few annotated samples. Some FSS methods try to address this issue by using the background in the support image(s) to help identify the background in the query image. This article proposes a method, QSR, that extracts the background from the query image itself.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Few-shot segmentation (FSS) aims to segment unseen classes using a few annotated samples. Typically, a prototype representing the foreground class is extracted from annotated support image(s) and is matched to features representing each pixel in the query image. However, models learnt in this way are insufficiently discriminatory, and often produce false positives: misclassifying background pixels as foreground. Some FSS methods try to address this issue by using the background in the support image(s) to help identify the background in the query image. However, the backgrounds of theses images is often quite distinct, and hence, the support image background information is uninformative. This article proposes a method, QSR, that extracts the background from the query image itself, and as a result is better able to discriminate between foreground and background features in the query image. This is achieved by modifying the training process to associate prototypes with class labels including known classes from the training data and latent classes representing unknown background objects. This class information is then used to extract a background prototype from the query image. To successfully associate prototypes with class labels and extract a background prototype that is capable of predicting a mask for the background regions of the image, the machinery for extracting and using foreground prototypes is induced to become more discriminative between different classes. Experiments for both 1-shot and 5-shot FSS on both the PASCAL-5i and COCO-20i datasets demonstrate that the proposed method results in a significant improvement in performance for the baseline methods it is applied to. As QSR operates only during training, these improved results are produced with no extra computational complexity during testing.

Related papers

• Is Foreground Prototype Sufficient? Few-Shot Medical Image Segmentation with Background-Fused Prototype [40.062825908232185]
  Few-shot Semantic(FSS)aim to adapt a pre-trained model to new classes with as few as a single labeled training sample per class. We present a new pluggable Background-fused prototype(Bro) for FSS in medical images. Bro incorporates this background with two pivot designs. Specifically, Feature Similarity(FeaC)initially reduces noise in the support image by employing feature cross-attention with the query image. We achieve this by a channel groups-based attention mechanism, where an adversarial structure encourages a coarse-to-fine fusion.
  arXiv  Detail & Related papers  (2024-12-04T02:51:22Z)
• Breaking Immutable: Information-Coupled Prototype Elaboration for Few-Shot Object Detection [15.079980293820137]
  We propose an Information-Coupled Prototype Elaboration (ICPE) method to generate specific and representative prototypes for each query image. Our method achieves state-of-the-art performance in almost all settings.
  arXiv  Detail & Related papers  (2022-11-27T10:33:11Z)
• Invariant Learning via Diffusion Dreamed Distribution Shifts [121.71383835729848]
  We propose a dataset called Diffusion Dreamed Distribution Shifts (D3S) D3S consists of synthetic images generated through StableDiffusion using text prompts and image guides obtained by pasting a sample foreground image onto a background template image. Due to the incredible photorealism of the diffusion model, our images are much closer to natural images than previous synthetic datasets.
  arXiv  Detail & Related papers  (2022-11-18T17:07:43Z)
• Scrape, Cut, Paste and Learn: Automated Dataset Generation Applied to Parcel Logistics [58.720142291102135]
  We present a fully automated pipeline to generate a synthetic dataset for instance segmentation in four steps. We first scrape images for the objects of interest from popular image search engines. We compare three different methods for image selection: Object-agnostic pre-processing, manual image selection and CNN-based image selection.
  arXiv  Detail & Related papers  (2022-10-18T12:49:04Z)
• On Background Bias in Deep Metric Learning [5.368313160283353]
  We analyze the influence of the image background on Deep Metric Learning models. We show that replacing the background of images during training with random background images alleviates this issue.
  arXiv  Detail & Related papers  (2022-10-04T13:57:39Z)
• Sparse Spatial Transformers for Few-Shot Learning [6.271261279657655]
  Learning from limited data is challenging because data scarcity leads to a poor generalization of the trained model. We propose a novel transformer-based neural network architecture called sparse spatial transformers. Our method finds task-relevant features and suppresses task-irrelevant features.
  arXiv  Detail & Related papers  (2021-09-27T10:36:32Z)
• Rectifying the Shortcut Learning of Background: Shared Object Concentration for Few-Shot Image Recognition [101.59989523028264]
  Few-Shot image classification aims to utilize pretrained knowledge learned from a large-scale dataset to tackle a series of downstream classification tasks. We propose COSOC, a novel Few-Shot Learning framework, to automatically figure out foreground objects at both pretraining and evaluation stage.
  arXiv  Detail & Related papers  (2021-07-16T07:46:41Z)
• SCNet: Enhancing Few-Shot Semantic Segmentation by Self-Contrastive Background Prototypes [56.387647750094466]
  Few-shot semantic segmentation aims to segment novel-class objects in a query image with only a few annotated examples. Most of advanced solutions exploit a metric learning framework that performs segmentation through matching each pixel to a learned foreground prototype. This framework suffers from biased classification due to incomplete construction of sample pairs with the foreground prototype only.
  arXiv  Detail & Related papers  (2021-04-19T11:21:47Z)
• Semantically Meaningful Class Prototype Learning for One-Shot Image Semantic Segmentation [58.96902899546075]
  One-shot semantic image segmentation aims to segment the object regions for the novel class with only one annotated image. Recent works adopt the episodic training strategy to mimic the expected situation at testing time. We propose to leverage the multi-class label information during the episodic training. It will encourage the network to generate more semantically meaningful features for each category.
  arXiv  Detail & Related papers  (2021-02-22T12:07:35Z)
• Distilling Localization for Self-Supervised Representation Learning [82.79808902674282]
  Contrastive learning has revolutionized unsupervised representation learning. Current contrastive models are ineffective at localizing the foreground object. We propose a data-driven approach for learning in variance to backgrounds.
  arXiv  Detail & Related papers  (2020-04-14T16:29:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.