Fine-Grained Prototypes Distillation for Few-Shot Object Detection
- URL: http://arxiv.org/abs/2401.07629v2
- Date: Tue, 12 Mar 2024 04:09:02 GMT
- Title: Fine-Grained Prototypes Distillation for Few-Shot Object Detection
- Authors: Zichen Wang, Bo Yang, Haonan Yue, Zhenghao Ma
- Abstract summary: Few-shot object detection (FSOD) aims at extending a generic detector for novel object detection with only a few training examples.
In general, methods based on meta-learning employ an additional support branch to encode novel examples into class prototypes.
New methods are required to capture the distinctive local context for more robust novel object detection.
- Score: 8.795211323408513
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Few-shot object detection (FSOD) aims at extending a generic detector for
novel object detection with only a few training examples. It attracts great
concerns recently due to the practical meanings. Meta-learning has been
demonstrated to be an effective paradigm for this task. In general, methods
based on meta-learning employ an additional support branch to encode novel
examples (a.k.a. support images) into class prototypes, which are then fused
with query branch to facilitate the model prediction. However, the class-level
prototypes are difficult to precisely generate, and they also lack detailed
information, leading to instability in performance.New methods are required to
capture the distinctive local context for more robust novel object detection.
To this end, we propose to distill the most representative support features
into fine-grained prototypes. These prototypes are then assigned into query
feature maps based on the matching results, modeling the detailed feature
relations between two branches. This process is realized by our Fine-Grained
Feature Aggregation (FFA) module. Moreover, in terms of high-level feature
fusion, we propose Balanced Class-Agnostic Sampling (B-CAS) strategy and
Non-Linear Fusion (NLF) module from differenct perspectives. They are
complementary to each other and depict the high-level feature relations more
effectively. Extensive experiments on PASCAL VOC and MS COCO benchmarks show
that our method sets a new state-of-the-art performance in most settings. Our
code is available at https://github.com/wangchen1801/FPD.
Related papers
- Open-Vocabulary Object Detection with Meta Prompt Representation and Instance Contrastive Optimization [63.66349334291372]
We propose a framework with Meta prompt and Instance Contrastive learning (MIC) schemes.
Firstly, we simulate a novel-class-emerging scenario to help the prompt that learns class and background prompts generalize to novel classes.
Secondly, we design an instance-level contrastive strategy to promote intra-class compactness and inter-class separation, which benefits generalization of the detector to novel class objects.
arXiv Detail & Related papers (2024-03-14T14:25:10Z) - Boosting Few-shot 3D Point Cloud Segmentation via Query-Guided
Enhancement [30.017448714419455]
This paper proposes a novel approach to improve point cloud few-shot segmentation (PC-FSS) models.
Unlike existing PC-FSS methods that directly utilize categorical information from support prototypes to recognize novel classes in query samples, our method identifies two critical aspects that substantially enhance model performance.
arXiv Detail & Related papers (2023-08-06T18:07:45Z) - Query Adaptive Few-Shot Object Detection with Heterogeneous Graph
Convolutional Networks [33.446875089255876]
Few-shot object detection (FSOD) aims to detect never-seen objects using few examples.
We propose a novel FSOD model using heterogeneous graph convolutional networks.
arXiv Detail & Related papers (2021-12-17T22:08:15Z) - APANet: Adaptive Prototypes Alignment Network for Few-Shot Semantic
Segmentation [56.387647750094466]
Few-shot semantic segmentation aims to segment novel-class objects in a given query image with only a few labeled support images.
Most advanced solutions exploit a metric learning framework that performs segmentation through matching each query feature to a learned class-specific prototype.
We present an adaptive prototype representation by introducing class-specific and class-agnostic prototypes.
arXiv Detail & Related papers (2021-11-24T04:38:37Z) - Dual Prototypical Contrastive Learning for Few-shot Semantic
Segmentation [55.339405417090084]
We propose a dual prototypical contrastive learning approach tailored to the few-shot semantic segmentation (FSS) task.
The main idea is to encourage the prototypes more discriminative by increasing inter-class distance while reducing intra-class distance in prototype feature space.
We demonstrate that the proposed dual contrastive learning approach outperforms state-of-the-art FSS methods on PASCAL-5i and COCO-20i datasets.
arXiv Detail & Related papers (2021-11-09T08:14:50Z) - Plug-and-Play Few-shot Object Detection with Meta Strategy and Explicit
Localization Inference [78.41932738265345]
This paper proposes a plug detector that can accurately detect the objects of novel categories without fine-tuning process.
We introduce two explicit inferences into the localization process to reduce its dependence on annotated data.
It shows a significant lead in both efficiency, precision, and recall under varied evaluation protocols.
arXiv Detail & Related papers (2021-10-26T03:09:57Z) - Prototype Completion for Few-Shot Learning [13.63424509914303]
Few-shot learning aims to recognize novel classes with few examples.
Pre-training based methods effectively tackle the problem by pre-training a feature extractor and then fine-tuning it through the nearest centroid based meta-learning.
We propose a novel prototype completion based meta-learning framework.
arXiv Detail & Related papers (2021-08-11T03:44:00Z) - Dynamic Relevance Learning for Few-Shot Object Detection [6.550840743803705]
We propose a dynamic relevance learning model, which utilizes the relationship between all support images and Region of Interest (RoI) on the query images to construct a dynamic graph convolutional network (GCN)
The proposed model achieves the best overall performance, which shows its effectiveness of learning more generalized features.
arXiv Detail & Related papers (2021-08-04T18:29:42Z) - MM-FSOD: Meta and metric integrated few-shot object detection [14.631208179789583]
We present an effective object detection framework (MM-FSOD) that integrates metric learning and meta-learning.
Our model is a class-agnostic detection model that can accurately recognize new categories, which are not appearing in training samples.
arXiv Detail & Related papers (2020-12-30T14:02:52Z) - Prior Guided Feature Enrichment Network for Few-Shot Segmentation [64.91560451900125]
State-of-the-art semantic segmentation methods require sufficient labeled data to achieve good results.
Few-shot segmentation is proposed to tackle this problem by learning a model that quickly adapts to new classes with a few labeled support samples.
Theses frameworks still face the challenge of generalization ability reduction on unseen classes due to inappropriate use of high-level semantic information.
arXiv Detail & Related papers (2020-08-04T10:41:32Z) - UniT: Unified Knowledge Transfer for Any-shot Object Detection and
Segmentation [52.487469544343305]
Methods for object detection and segmentation rely on large scale instance-level annotations for training.
We propose an intuitive and unified semi-supervised model that is applicable to a range of supervision.
arXiv Detail & Related papers (2020-06-12T22:45:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.