OSR-ViT: A Simple and Modular Framework for Open-Set Object Detection and Discovery

• URL: http://arxiv.org/abs/2404.10865v1
• Date: Tue, 16 Apr 2024 19:29:27 GMT
• Title: OSR-ViT: A Simple and Modular Framework for Open-Set Object Detection and Discovery
• Authors: Matthew Inkawhich, Nathan Inkawhich, Hao Yang, Jingyang Zhang, Randolph Linderman, Yiran Chen,
• Abstract summary: We present a new task called Open-Set Object Detection and Discovery (OSODD) We propose the Open-Set Regions with ViT features (OSR-ViT) detection framework. OSR-ViT combines a class-agnostic proposal network with a powerful ViT-based classifier.
• Score: 16.055210504552406
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: An object detector's ability to detect and flag \textit{novel} objects during open-world deployments is critical for many real-world applications. Unfortunately, much of the work in open object detection today is disjointed and fails to adequately address applications that prioritize unknown object recall \textit{in addition to} known-class accuracy. To close this gap, we present a new task called Open-Set Object Detection and Discovery (OSODD) and as a solution propose the Open-Set Regions with ViT features (OSR-ViT) detection framework. OSR-ViT combines a class-agnostic proposal network with a powerful ViT-based classifier. Its modular design simplifies optimization and allows users to easily swap proposal solutions and feature extractors to best suit their application. Using our multifaceted evaluation protocol, we show that OSR-ViT obtains performance levels that far exceed state-of-the-art supervised methods. Our method also excels in low-data settings, outperforming supervised baselines using a fraction of the training data.

Related papers

• Learning to Borrow Features for Improved Detection of Small Objects in Single-Shot Detectors [0.0]
  We propose a novel framework that enables small object representations to "borrow" discriminative features from larger, semantically richer instances within the same class. Our approach significantly boosts small object detection accuracy over baseline methods, offering a promising direction for robust object detection in complex visual environments.
  arXiv  Detail & Related papers  (2025-04-30T01:18:33Z)
• MB-ORES: A Multi-Branch Object Reasoner for Visual Grounding in Remote Sensing [0.08192907805418585]
  We propose a unified framework that integrates object detection (OD) and visual grounding (VG) for remote sensing (RS) imagery. Our model demonstrates superior performance on the OPT-RSVG and DIOR-RSVG datasets.
  arXiv  Detail & Related papers  (2025-03-31T15:36:41Z)
• OpenEarthSensing: Large-Scale Fine-Grained Benchmark for Open-World Remote Sensing [57.050679160659705]
  We introduce textbfOpenEarthSensing (OES), a large-scale fine-grained benchmark for open-world remote sensing.<n>OES includes 189 scene and object categories, covering the vast majority of potential semantic shifts that may occur in the real world.
  arXiv  Detail & Related papers  (2025-02-28T02:49:52Z)
• Open-World Object Detection with Instance Representation Learning [1.8749305679160366]
  We propose a method to train an object detector that can both detect novel objects and extract semantically rich features in open-world conditions. Our method learns a robust and generalizable feature space, outperforming other OWOD-based feature extraction methods.
  arXiv  Detail & Related papers  (2024-09-24T13:13:34Z)
• Weakly Supervised Open-Vocabulary Object Detection [31.605276665964787]
  We propose a novel weakly supervised open-vocabulary object detection framework, namely WSOVOD, to extend traditional WSOD. To achieve this, we explore three vital strategies, including dataset-level feature adaptation, image-level salient object localization, and region-level vision-language alignment.
  arXiv  Detail & Related papers  (2023-12-19T18:59:53Z)
• Weakly-supervised Contrastive Learning for Unsupervised Object Discovery [52.696041556640516]
  Unsupervised object discovery is promising due to its ability to discover objects in a generic manner. We design a semantic-guided self-supervised learning model to extract high-level semantic features from images. We introduce Principal Component Analysis (PCA) to localize object regions.
  arXiv  Detail & Related papers  (2023-07-07T04:03:48Z)
• SalienDet: A Saliency-based Feature Enhancement Algorithm for Object Detection for Autonomous Driving [160.57870373052577]
  We propose a saliency-based OD algorithm (SalienDet) to detect unknown objects. Our SalienDet utilizes a saliency-based algorithm to enhance image features for object proposal generation. We design a dataset relabeling approach to differentiate the unknown objects from all objects in training sample set to achieve Open-World Detection.
  arXiv  Detail & Related papers  (2023-05-11T16:19:44Z)
• Open-Set Object Detection Using Classification-free Object Proposal and Instance-level Contrastive Learning [25.935629339091697]
  Open-set object detection (OSOD) is a promising direction to handle the problem consisting of two subtasks: objects and background separation, and open-set object classification. We present Openset RCNN to address the challenging OSOD. We show that our Openset RCNN can endow the robot with an open-set perception ability to support robotic rearrangement tasks in cluttered environments.
  arXiv  Detail & Related papers  (2022-11-21T15:00:04Z)
• Multi-modal Transformers Excel at Class-agnostic Object Detection [105.10403103027306]
  We argue that existing methods lack a top-down supervision signal governed by human-understandable semantics. We develop an efficient and flexible MViT architecture using multi-scale feature processing and deformable self-attention. We show the significance of MViT proposals in a diverse range of applications.
  arXiv  Detail & Related papers  (2021-11-22T18:59:29Z)
• Discovery-and-Selection: Towards Optimal Multiple Instance Learning for Weakly Supervised Object Detection [86.86602297364826]
  We propose a discoveryand-selection approach fused with multiple instance learning (DS-MIL) Our proposed DS-MIL approach can consistently improve the baselines, reporting state-of-the-art performance.
  arXiv  Detail & Related papers  (2021-10-18T07:06:57Z)
• Learning Open-World Object Proposals without Learning to Classify [110.30191531975804]
  We propose a classification-free Object Localization Network (OLN) which estimates the objectness of each region purely by how well the location and shape of a region overlaps with any ground-truth object. This simple strategy learns generalizable objectness and outperforms existing proposals on cross-category generalization.
  arXiv  Detail & Related papers  (2021-08-15T14:36:02Z)
• Robust Object Detection via Instance-Level Temporal Cycle Confusion [89.1027433760578]
  We study the effectiveness of auxiliary self-supervised tasks to improve the out-of-distribution generalization of object detectors. Inspired by the principle of maximum entropy, we introduce a novel self-supervised task, instance-level temporal cycle confusion (CycConf) For each object, the task is to find the most different object proposals in the adjacent frame in a video and then cycle back to itself for self-supervision.
  arXiv  Detail & Related papers  (2021-04-16T21:35:08Z)
• Uncertainty for Identifying Open-Set Errors in Visual Object Detection [31.533136658421892]
  GMM-Det is a real-time method for extracting uncertainty from object detectors to identify and reject open-set errors. We show that GMM-Det consistently outperforms existing uncertainty techniques for identifying and rejecting open-set detections.
  arXiv  Detail & Related papers  (2021-04-03T07:12:31Z)
• Distilling Knowledge from Refinement in Multiple Instance Detection Networks [0.0]
  Weakly supervised object detection (WSOD) aims to tackle the object detection problem using only labeled image categories as supervision. We present an adaptive supervision aggregation function that dynamically changes the aggregation criteria for selecting boxes related to one of the ground-truth classes, background, or even ignored during the generation of each refinement module supervision.
  arXiv  Detail & Related papers  (2020-04-23T02:49:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.