Open-Set Object Detection Using Classification-free Object Proposal and Instance-level Contrastive Learning

• URL: http://arxiv.org/abs/2211.11530v2
• Date: Mon, 4 Dec 2023 02:58:37 GMT
• Title: Open-Set Object Detection Using Classification-free Object Proposal and Instance-level Contrastive Learning
• Authors: Zhongxiang Zhou, Yifei Yang, Yue Wang, Rong Xiong
• Abstract summary: Open-set object detection (OSOD) is a promising direction to handle the problem consisting of two subtasks: objects and background separation, and open-set object classification. We present Openset RCNN to address the challenging OSOD. We show that our Openset RCNN can endow the robot with an open-set perception ability to support robotic rearrangement tasks in cluttered environments.
• Score: 25.935629339091697
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Detecting both known and unknown objects is a fundamental skill for robot manipulation in unstructured environments. Open-set object detection (OSOD) is a promising direction to handle the problem consisting of two subtasks: objects and background separation, and open-set object classification. In this paper, we present Openset RCNN to address the challenging OSOD. To disambiguate unknown objects and background in the first subtask, we propose to use classification-free region proposal network (CF-RPN) which estimates the objectness score of each region purely using cues from object's location and shape preventing overfitting to the training categories. To identify unknown objects in the second subtask, we propose to represent them using the complementary region of known categories in a latent space which is accomplished by a prototype learning network (PLN). PLN performs instance-level contrastive learning to encode proposals to a latent space and builds a compact region centering with a prototype for each known category. Further, we note that the detection performance of unknown objects can not be unbiasedly evaluated on the situation that commonly used object detection datasets are not fully annotated. Thus, a new benchmark is introduced by reorganizing GraspNet-1billion, a robotic grasp pose detection dataset with complete annotation. Extensive experiments demonstrate the merits of our method. We finally show that our Openset RCNN can endow the robot with an open-set perception ability to support robotic rearrangement tasks in cluttered environments. More details can be found in https://sites.google.com/view/openset-rcnn/

Related papers

• Exploiting Unlabeled Data with Multiple Expert Teachers for Open Vocabulary Aerial Object Detection and Its Orientation Adaptation [58.37525311718006]
  We put forth a novel formulation of the aerial object detection problem, namely open-vocabulary aerial object detection (OVAD) We propose CastDet, a CLIP-activated student-teacher detection framework that serves as the first OVAD detector specifically designed for the challenging aerial scenario. Our framework integrates a robust localization teacher along with several box selection strategies to generate high-quality proposals for novel objects.
  arXiv  Detail & Related papers  (2024-11-04T12:59:13Z)
• Few-shot Object Detection in Remote Sensing: Lifting the Curse of Incompletely Annotated Novel Objects [23.171410277239534]
  We propose a self-training-based FSOD (ST-FSOD) approach to object detection. Our proposed method outperforms the state-of-the-art in various FSOD settings by a large margin.
  arXiv  Detail & Related papers  (2023-09-19T13:00:25Z)
• Weakly-supervised Contrastive Learning for Unsupervised Object Discovery [52.696041556640516]
  Unsupervised object discovery is promising due to its ability to discover objects in a generic manner. We design a semantic-guided self-supervised learning model to extract high-level semantic features from images. We introduce Principal Component Analysis (PCA) to localize object regions.
  arXiv  Detail & Related papers  (2023-07-07T04:03:48Z)
• SalienDet: A Saliency-based Feature Enhancement Algorithm for Object Detection for Autonomous Driving [160.57870373052577]
  We propose a saliency-based OD algorithm (SalienDet) to detect unknown objects. Our SalienDet utilizes a saliency-based algorithm to enhance image features for object proposal generation. We design a dataset relabeling approach to differentiate the unknown objects from all objects in training sample set to achieve Open-World Detection.
  arXiv  Detail & Related papers  (2023-05-11T16:19:44Z)
• Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection [76.5120397167247]
  We present an open-set object detector, called Grounding DINO, by marrying Transformer-based detector DINO with grounded pre-training. The key solution of open-set object detection is introducing language to a closed-set detector for open-set concept generalization. Grounding DINO performs remarkably well on all three settings, including benchmarks on COCO, LVIS, ODinW, and RefCOCO/+/g.
  arXiv  Detail & Related papers  (2023-03-09T18:52:16Z)
• Open World DETR: Transformer based Open World Object Detection [60.64535309016623]
  We propose a two-stage training approach named Open World DETR for open world object detection based on Deformable DETR. We fine-tune the class-specific components of the model with a multi-view self-labeling strategy and a consistency constraint. Our proposed method outperforms other state-of-the-art open world object detection methods by a large margin.
  arXiv  Detail & Related papers  (2022-12-06T13:39:30Z)
• PROB: Probabilistic Objectness for Open World Object Detection [15.574535196804042]
  Open World Object Detection (OWOD) is a new computer vision task that bridges the gap between classic object detection (OD) benchmarks and object detection in the real world. We introduce a novel probabilistic framework for objectness estimation, where we alternate between probability distribution estimation and objectness likelihood of known objects. The resulting Probabilistic Objectness transformer-based open-world detector, PROB, integrates our framework into traditional object detection models.
  arXiv  Detail & Related papers  (2022-12-02T20:04:24Z)
• Towards Open-Set Object Detection and Discovery [38.81806249664884]
  We present a new task, namely Open-Set Object Detection and Discovery (OSODD) We propose a two-stage method that first uses an open-set object detector to predict both known and unknown objects. Then, we study the representation of predicted objects in an unsupervised manner and discover new categories from the set of unknown objects.
  arXiv  Detail & Related papers  (2022-04-12T08:07:01Z)
• Learning Open-World Object Proposals without Learning to Classify [110.30191531975804]
  We propose a classification-free Object Localization Network (OLN) which estimates the objectness of each region purely by how well the location and shape of a region overlaps with any ground-truth object. This simple strategy learns generalizable objectness and outperforms existing proposals on cross-category generalization.
  arXiv  Detail & Related papers  (2021-08-15T14:36:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.