Breaking Immutable: Information-Coupled Prototype Elaboration for Few-Shot Object Detection

• URL: http://arxiv.org/abs/2211.14782v1
• Date: Sun, 27 Nov 2022 10:33:11 GMT
• Title: Breaking Immutable: Information-Coupled Prototype Elaboration for Few-Shot Object Detection
• Authors: Xiaonan Lu, Wenhui Diao, Yongqiang Mao, Junxi Li, Peijin Wang, Xian Sun, Kun Fu
• Abstract summary: We propose an Information-Coupled Prototype Elaboration (ICPE) method to generate specific and representative prototypes for each query image. Our method achieves state-of-the-art performance in almost all settings.
• Score: 15.079980293820137
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Few-shot object detection, expecting detectors to detect novel classes with a few instances, has made conspicuous progress. However, the prototypes extracted by existing meta-learning based methods still suffer from insufficient representative information and lack awareness of query images, which cannot be adaptively tailored to different query images. Firstly, only the support images are involved for extracting prototypes, resulting in scarce perceptual information of query images. Secondly, all pixels of all support images are treated equally when aggregating features into prototype vectors, thus the salient objects are overwhelmed by the cluttered background. In this paper, we propose an Information-Coupled Prototype Elaboration (ICPE) method to generate specific and representative prototypes for each query image. Concretely, a conditional information coupling module is introduced to couple information from the query branch to the support branch, strengthening the query-perceptual information in support features. Besides, we design a prototype dynamic aggregation module that dynamically adjusts intra-image and inter-image aggregation weights to highlight the salient information useful for detecting query images. Experimental results on both Pascal VOC and MS COCO demonstrate that our method achieves state-of-the-art performance in almost all settings.

Related papers

• Correlation Weighted Prototype-based Self-Supervised One-Shot Segmentation of Medical Images [12.365801596593936]
  Medical image segmentation is one of the domains where sufficient annotated data is not available. We propose a prototype-based self-supervised one-way one-shot learning framework using pseudo-labels generated from superpixels. We show that the proposed simple but potent framework performs at par with the state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-08-12T15:38:51Z)
• Fine-Grained Prototypes Distillation for Few-Shot Object Detection [8.795211323408513]
  Few-shot object detection (FSOD) aims at extending a generic detector for novel object detection with only a few training examples. In general, methods based on meta-learning employ an additional support branch to encode novel examples into class prototypes. New methods are required to capture the distinctive local context for more robust novel object detection.
  arXiv  Detail & Related papers  (2024-01-15T12:12:48Z)
• With a Little Help from your own Past: Prototypical Memory Networks for Image Captioning [47.96387857237473]
  We devise a network which can perform attention over activations obtained while processing other training samples. Our memory models the distribution of past keys and values through the definition of prototype vectors. We demonstrate that our proposal can increase the performance of an encoder-decoder Transformer by 3.7 CIDEr points both when training in cross-entropy only and when fine-tuning with self-critical sequence training.
  arXiv  Detail & Related papers  (2023-08-23T18:53:00Z)
• Holistic Prototype Attention Network for Few-Shot VOS [74.25124421163542]
  Few-shot video object segmentation (FSVOS) aims to segment dynamic objects of unseen classes by resorting to a small set of support images. We propose a holistic prototype attention network (HPAN) for advancing FSVOS.
  arXiv  Detail & Related papers  (2023-07-16T03:48:57Z)
• MIANet: Aggregating Unbiased Instance and General Information for Few-Shot Semantic Segmentation [6.053853367809978]
  Existing few-shot segmentation methods are based on the meta-learning strategy and extract instance knowledge from a support set. We propose a multi-information aggregation network (MIANet) that effectively leverages the general knowledge, i.e., semantic word embeddings, and instance information for accurate segmentation. Experiments on PASCAL-5i and COCO-20i show that MIANet yields superior performance and set a new state-of-the-art.
  arXiv  Detail & Related papers  (2023-05-23T09:36:27Z)
• Prototype as Query for Few Shot Semantic Segmentation [7.380266341356485]
  Few-shot Semantic (FSS) was proposed to segment unseen classes in a query image, referring to only a few examples named support images. We propose a framework built upon Transformer termed as ProtoFormer to fully capture spatial details in query features.
  arXiv  Detail & Related papers  (2022-11-27T08:41:50Z)
• Intermediate Prototype Mining Transformer for Few-Shot Semantic Segmentation [119.51445225693382]
  Few-shot semantic segmentation aims to segment the target objects in query under the condition of a few annotated support images. We introduce an intermediate prototype for mining both deterministic category information from the support and adaptive category knowledge from the query. In each IPMT layer, we propagate the object information in both support and query features to the prototype and then use it to activate the query feature map.
  arXiv  Detail & Related papers  (2022-10-13T06:45:07Z)
• Self-supervised Image-specific Prototype Exploration for Weakly Supervised Semantic Segmentation [72.33139350241044]
  Weakly Supervised Semantic COCO (WSSS) based on image-level labels has attracted much attention due to low annotation costs. We propose a Self-supervised Image-specific Prototype Exploration (SIPE) that consists of an Image-specific Prototype Exploration (IPE) and a General-Specific Consistency (GSC) loss. Our SIPE achieves new state-of-the-art performance using only image-level labels.
  arXiv  Detail & Related papers  (2022-03-06T09:01:03Z)
• Semantically Meaningful Class Prototype Learning for One-Shot Image Semantic Segmentation [58.96902899546075]
  One-shot semantic image segmentation aims to segment the object regions for the novel class with only one annotated image. Recent works adopt the episodic training strategy to mimic the expected situation at testing time. We propose to leverage the multi-class label information during the episodic training. It will encourage the network to generate more semantically meaningful features for each category.
  arXiv  Detail & Related papers  (2021-02-22T12:07:35Z)
• Prototype Mixture Models for Few-shot Semantic Segmentation [50.866870384596446]
  Few-shot segmentation is challenging because objects within the support and query images could significantly differ in appearance and pose. We propose prototype mixture models (PMMs), which correlate diverse image regions with multiple prototypes to enforce the prototype-based semantic representation. PMMs improve 5-shot segmentation performance on MS-COCO by up to 5.82% with only a moderate cost for model size and inference speed.
  arXiv  Detail & Related papers  (2020-08-10T04:33:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.