Open World DETR: Transformer based Open World Object Detection
- URL: http://arxiv.org/abs/2212.02969v1
- Date: Tue, 6 Dec 2022 13:39:30 GMT
- Title: Open World DETR: Transformer based Open World Object Detection
- Authors: Na Dong, Yongqiang Zhang, Mingli Ding, Gim Hee Lee
- Abstract summary: We propose a two-stage training approach named Open World DETR for open world object detection based on Deformable DETR.
We fine-tune the class-specific components of the model with a multi-view self-labeling strategy and a consistency constraint.
Our proposed method outperforms other state-of-the-art open world object detection methods by a large margin.
- Score: 60.64535309016623
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Open world object detection aims at detecting objects that are absent in the
object classes of the training data as unknown objects without explicit
supervision. Furthermore, the exact classes of the unknown objects must be
identified without catastrophic forgetting of the previous known classes when
the corresponding annotations of unknown objects are given incrementally. In
this paper, we propose a two-stage training approach named Open World DETR for
open world object detection based on Deformable DETR. In the first stage, we
pre-train a model on the current annotated data to detect objects from the
current known classes, and concurrently train an additional binary classifier
to classify predictions into foreground or background classes. This helps the
model to build an unbiased feature representations that can facilitate the
detection of unknown classes in subsequent process. In the second stage, we
fine-tune the class-specific components of the model with a multi-view
self-labeling strategy and a consistency constraint. Furthermore, we alleviate
catastrophic forgetting when the annotations of the unknown classes becomes
available incrementally by using knowledge distillation and exemplar replay.
Experimental results on PASCAL VOC and MS-COCO show that our proposed method
outperforms other state-of-the-art open world object detection methods by a
large margin.
Related papers
- Open-World Object Detection with Instance Representation Learning [1.8749305679160366]
We propose a method to train an object detector that can both detect novel objects and extract semantically rich features in open-world conditions.
Our method learns a robust and generalizable feature space, outperforming other OWOD-based feature extraction methods.
arXiv Detail & Related papers (2024-09-24T13:13:34Z) - Unsupervised Recognition of Unknown Objects for Open-World Object
Detection [28.787586991713535]
Open-World Object Detection (OWOD) extends object detection problem to a realistic and dynamic scenario.
Current OWOD models, such as ORE and OW-DETR, focus on pseudo-labeling regions with high objectness scores as unknowns.
This paper proposes a novel approach that learns an unsupervised discriminative model to recognize true unknown objects.
arXiv Detail & Related papers (2023-08-31T08:17:29Z) - Weakly-supervised Contrastive Learning for Unsupervised Object Discovery [52.696041556640516]
Unsupervised object discovery is promising due to its ability to discover objects in a generic manner.
We design a semantic-guided self-supervised learning model to extract high-level semantic features from images.
We introduce Principal Component Analysis (PCA) to localize object regions.
arXiv Detail & Related papers (2023-07-07T04:03:48Z) - Open-World Object Detection via Discriminative Class Prototype Learning [4.055884768256164]
Open-world object detection (OWOD) is a challenging problem that combines object detection with incremental learning and open-set learning.
We propose a novel and efficient OWOD solution from a prototype perspective, which we call OCPL: Open-world object detection via discnative OCPL: Open-world object detection via discriminative OCPL: Open-world object detection via discriminative OCPL: Open-world object detection via discriminative OCPL: Open-world object detection via discriminative OCPL: Open-world object detection via discriminative OCPL: Open-world object detection via
arXiv Detail & Related papers (2023-02-23T03:05:04Z) - Detecting the unknown in Object Detection [20.84221126313118]
We propose a novel training strategy, called UNKAD, able to predict unknown objects without requiring any annotation.
UNKAD first identifies and pseudo-labels unknown objects and then uses the pseudo-annotations to train an additional unknown class.
While UNKAD can directly detect unknown objects, we further combine it with previous unknown detection techniques, showing that it improves their performance at no costs.
arXiv Detail & Related papers (2022-08-24T16:27:38Z) - Towards Open-Set Object Detection and Discovery [38.81806249664884]
We present a new task, namely Open-Set Object Detection and Discovery (OSODD)
We propose a two-stage method that first uses an open-set object detector to predict both known and unknown objects.
Then, we study the representation of predicted objects in an unsupervised manner and discover new categories from the set of unknown objects.
arXiv Detail & Related papers (2022-04-12T08:07:01Z) - Contrastive Object Detection Using Knowledge Graph Embeddings [72.17159795485915]
We compare the error statistics of the class embeddings learned from a one-hot approach with semantically structured embeddings from natural language processing or knowledge graphs.
We propose a knowledge-embedded design for keypoint-based and transformer-based object detection architectures.
arXiv Detail & Related papers (2021-12-21T17:10:21Z) - OW-DETR: Open-world Detection Transformer [90.56239673123804]
We introduce a novel end-to-end transformer-based framework, OW-DETR, for open-world object detection.
OW-DETR comprises three dedicated components namely, attention-driven pseudo-labeling, novelty classification and objectness scoring.
Our model outperforms the recently introduced OWOD approach, ORE, with absolute gains ranging from 1.8% to 3.3% in terms of unknown recall.
arXiv Detail & Related papers (2021-12-02T18:58:30Z) - Learning Open-World Object Proposals without Learning to Classify [110.30191531975804]
We propose a classification-free Object Localization Network (OLN) which estimates the objectness of each region purely by how well the location and shape of a region overlaps with any ground-truth object.
This simple strategy learns generalizable objectness and outperforms existing proposals on cross-category generalization.
arXiv Detail & Related papers (2021-08-15T14:36:02Z) - Synthesizing the Unseen for Zero-shot Object Detection [72.38031440014463]
We propose to synthesize visual features for unseen classes, so that the model learns both seen and unseen objects in the visual domain.
We use a novel generative model that uses class-semantics to not only generate the features but also to discriminatively separate them.
arXiv Detail & Related papers (2020-10-19T12:36:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.