Detecting the open-world objects with the help of the Brain

• URL: http://arxiv.org/abs/2303.11623v1
• Date: Tue, 21 Mar 2023 06:44:02 GMT
• Title: Detecting the open-world objects with the help of the Brain
• Authors: Shuailei Ma, Yuefeng Wang, Ying Wei, Peihao Chen, Zhixiang Ye, Jiaqi Fan, Enming Zhang, Thomas H. Li
• Abstract summary: Open World Object Detection (OWOD) is a novel computer vision task with a considerable challenge. OWOD algorithms are expected to detect unseen/unknown objects and incrementally learn them. We propose leveraging the VL as the Brain'' of the open-world detector by simply generating unknown labels.
• Score: 20.00772846521719
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Open World Object Detection (OWOD) is a novel computer vision task with a considerable challenge, bridging the gap between classic object detection (OD) benchmarks and real-world object detection. In addition to detecting and classifying seen/known objects, OWOD algorithms are expected to detect unseen/unknown objects and incrementally learn them. The natural instinct of humans to identify unknown objects in their environments mainly depends on their brains' knowledge base. It is difficult for a model to do this only by learning from the annotation of several tiny datasets. The large pre-trained grounded language-image models - VL (\ie GLIP) have rich knowledge about the open world but are limited to the text prompt. We propose leveraging the VL as the ``Brain'' of the open-world detector by simply generating unknown labels. Leveraging it is non-trivial because the unknown labels impair the model's learning of known objects. In this paper, we alleviate these problems by proposing the down-weight loss function and decoupled detection structure. Moreover, our detector leverages the ``Brain'' to learn novel objects beyond VL through our pseudo-labeling scheme.

Related papers

• Open-World Object Detection with Instance Representation Learning [1.8749305679160366]
  We propose a method to train an object detector that can both detect novel objects and extract semantically rich features in open-world conditions. Our method learns a robust and generalizable feature space, outperforming other OWOD-based feature extraction methods.
  arXiv  Detail & Related papers  (2024-09-24T13:13:34Z)
• Learning Background Prompts to Discover Implicit Knowledge for Open Vocabulary Object Detection [101.15777242546649]
  Open vocabulary object detection (OVD) aims at seeking an optimal object detector capable of recognizing objects from both base and novel categories. Recent advances leverage knowledge distillation to transfer insightful knowledge from pre-trained large-scale vision-language models to the task of object detection. We present a novel OVD framework termed LBP to propose learning background prompts to harness explored implicit background knowledge.
  arXiv  Detail & Related papers  (2024-06-01T17:32:26Z)
• Unsupervised Recognition of Unknown Objects for Open-World Object Detection [28.787586991713535]
  Open-World Object Detection (OWOD) extends object detection problem to a realistic and dynamic scenario. Current OWOD models, such as ORE and OW-DETR, focus on pseudo-labeling regions with high objectness scores as unknowns. This paper proposes a novel approach that learns an unsupervised discriminative model to recognize true unknown objects.
  arXiv  Detail & Related papers  (2023-08-31T08:17:29Z)
• Weakly-supervised Contrastive Learning for Unsupervised Object Discovery [52.696041556640516]
  Unsupervised object discovery is promising due to its ability to discover objects in a generic manner. We design a semantic-guided self-supervised learning model to extract high-level semantic features from images. We introduce Principal Component Analysis (PCA) to localize object regions.
  arXiv  Detail & Related papers  (2023-07-07T04:03:48Z)
• SalienDet: A Saliency-based Feature Enhancement Algorithm for Object Detection for Autonomous Driving [160.57870373052577]
  We propose a saliency-based OD algorithm (SalienDet) to detect unknown objects. Our SalienDet utilizes a saliency-based algorithm to enhance image features for object proposal generation. We design a dataset relabeling approach to differentiate the unknown objects from all objects in training sample set to achieve Open-World Detection.
  arXiv  Detail & Related papers  (2023-05-11T16:19:44Z)
• Open World DETR: Transformer based Open World Object Detection [60.64535309016623]
  We propose a two-stage training approach named Open World DETR for open world object detection based on Deformable DETR. We fine-tune the class-specific components of the model with a multi-view self-labeling strategy and a consistency constraint. Our proposed method outperforms other state-of-the-art open world object detection methods by a large margin.
  arXiv  Detail & Related papers  (2022-12-06T13:39:30Z)
• Learning by Asking Questions for Knowledge-based Novel Object Recognition [64.55573343404572]
  In real-world object recognition, there are numerous object classes to be recognized. Conventional image recognition based on supervised learning can only recognize object classes that exist in the training data, and thus has limited applicability in the real world. Inspired by this, we study a framework for acquiring external knowledge through question generation that would help the model instantly recognize novel objects. Our pipeline consists of two components: the Object-based object recognition, and the Question Generator, which generates knowledge-aware questions to acquire novel knowledge.
  arXiv  Detail & Related papers  (2022-10-12T02:51:58Z)
• More Practical Scenario of Open-set Object Detection: Open at Category Level and Closed at Super-category Level [23.98839374194848]
  Open-set object detection (OSOD) has recently attracted considerable attention. We first point out that the scenario of OSOD considered in recent studies, which considers an unlimited variety of unknown objects, has a fundamental issue. This issue leads to difficulty with the evaluation of methods' performance on unknown object detection.
  arXiv  Detail & Related papers  (2022-07-20T09:28:51Z)
• Towards Open-Set Object Detection and Discovery [38.81806249664884]
  We present a new task, namely Open-Set Object Detection and Discovery (OSODD) We propose a two-stage method that first uses an open-set object detector to predict both known and unknown objects. Then, we study the representation of predicted objects in an unsupervised manner and discover new categories from the set of unknown objects.
  arXiv  Detail & Related papers  (2022-04-12T08:07:01Z)
• Towards Open World Object Detection [68.79678648726416]
  ORE: Open World Object Detector is based on contrastive clustering and energy based unknown identification. We find that identifying and characterizing unknown instances helps to reduce confusion in an incremental object detection setting.
  arXiv  Detail & Related papers  (2021-03-03T18:58:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.