YOLO-UniOW: Efficient Universal Open-World Object Detection

• URL: http://arxiv.org/abs/2412.20645v1
• Date: Mon, 30 Dec 2024 01:34:14 GMT
• Title: YOLO-UniOW: Efficient Universal Open-World Object Detection
• Authors: Lihao Liu, Juexiao Feng, Hui Chen, Ao Wang, Lin Song, Jungong Han, Guiguang Ding,
• Abstract summary: We introduce Universal Open-World Object Detection (Uni-OWD), a new paradigm that unifies open-vocabulary and open-world object detection tasks. YOLO-UniOW incorporates Adaptive Decision Learning to replace computationally expensive cross-modality fusion with lightweight alignment in the CLIP latent space. Experiments validate the superiority of YOLO-UniOW, achieving 34.6 AP and 30.0 APr with an inference speed of 69.6 FPS.
• Score: 63.71512991320627
• License:
• Abstract: Traditional object detection models are constrained by the limitations of closed-set datasets, detecting only categories encountered during training. While multimodal models have extended category recognition by aligning text and image modalities, they introduce significant inference overhead due to cross-modality fusion and still remain restricted by predefined vocabulary, leaving them ineffective at handling unknown objects in open-world scenarios. In this work, we introduce Universal Open-World Object Detection (Uni-OWD), a new paradigm that unifies open-vocabulary and open-world object detection tasks. To address the challenges of this setting, we propose YOLO-UniOW, a novel model that advances the boundaries of efficiency, versatility, and performance. YOLO-UniOW incorporates Adaptive Decision Learning to replace computationally expensive cross-modality fusion with lightweight alignment in the CLIP latent space, achieving efficient detection without compromising generalization. Additionally, we design a Wildcard Learning strategy that detects out-of-distribution objects as "unknown" while enabling dynamic vocabulary expansion without the need for incremental learning. This design empowers YOLO-UniOW to seamlessly adapt to new categories in open-world environments. Extensive experiments validate the superiority of YOLO-UniOW, achieving achieving 34.6 AP and 30.0 APr on LVIS with an inference speed of 69.6 FPS. The model also sets benchmarks on M-OWODB, S-OWODB, and nuScenes datasets, showcasing its unmatched performance in open-world object detection. Code and models are available at https://github.com/THU-MIG/YOLO-UniOW.

Related papers

• Oriented Tiny Object Detection: A Dataset, Benchmark, and Dynamic Unbiased Learning [51.170479006249195]
  We introduce a new dataset, benchmark, and a dynamic coarse-to-fine learning scheme in this study. Our proposed dataset, AI-TOD-R, features the smallest object sizes among all oriented object detection datasets. We present a benchmark spanning a broad range of detection paradigms, including both fully-supervised and label-efficient approaches.
  arXiv  Detail & Related papers  (2024-12-16T09:14:32Z)
• From Open Vocabulary to Open World: Teaching Vision Language Models to Detect Novel Objects [0.6262268096839562]
  Recent works on open vocabulary object detection (OVD) enable the detection of objects defined by an unbounded vocabulary. OVD relies on accurate prompts provided by an ''oracle'', which limits their use in critical applications such as driving scene perception. We propose a framework that enables OVD models to operate in open world settings, by identifying and incrementally learning novel objects.
  arXiv  Detail & Related papers  (2024-11-27T10:33:51Z)
• Mamba-YOLO-World: Marrying YOLO-World with Mamba for Open-Vocabulary Detection [18.65107742085838]
  We present Mamba-YOLO-World, a novel YOLO-based OVD model employing the proposed MambaFusion Path Aggregation Network (MambaFusion-PAN) as its neck architecture. Specifically, we introduce an innovative State Space Model-based feature fusion mechanism consisting of a Parallel-Guided Selective Scan algorithm and a Serial-Guided Selective Scan algorithm with linear complexity and globally guided receptive fields. Experiments demonstrate that our model outperforms the original YOLO-World on the COCO and LVIS benchmarks in both zero-shot and fine-tuning settings.
  arXiv  Detail & Related papers  (2024-09-13T03:23:52Z)
• GMFL-Net: A Global Multi-geometric Feature Learning Network for Repetitive Action Counting [4.117416395116726]
  We propose a simple but efficient Global Multi-geometric Feature Learning Network (GMFL-Net) Specifically, we design a MIA-Module that aims to improve information representation by fusing multi-geometric features. We also design a GBFL-Module that enhances the inter-dependencies between point-wise and channel-wise elements.
  arXiv  Detail & Related papers  (2024-08-31T02:18:26Z)
• YOLO-World: Real-Time Open-Vocabulary Object Detection [87.08732047660058]
  We introduce YOLO-World, an innovative approach that enhances YOLO with open-vocabulary detection capabilities. Our method excels in detecting a wide range of objects in a zero-shot manner with high efficiency. YOLO-World achieves 35.4 AP with 52.0 FPS on V100, which outperforms many state-of-the-art methods in terms of both accuracy and speed.
  arXiv  Detail & Related papers  (2024-01-30T18:59:38Z)
• Aligning and Prompting Everything All at Once for Universal Visual Perception [79.96124061108728]
  APE is a universal visual perception model for aligning and prompting everything all at once in an image to perform diverse tasks. APE advances the convergence of detection and grounding by reformulating language-guided grounding as open-vocabulary detection. Experiments on over 160 datasets demonstrate that APE outperforms state-of-the-art models.
  arXiv  Detail & Related papers  (2023-12-04T18:59:50Z)
• Towards Open-Ended Visual Recognition with Large Language Model [27.56182473356992]
  We introduce the OmniScient Model (OSM), a novel Large Language Model (LLM) based mask classifier. OSM predicts class labels in a generative manner, thus removing the supply of class names during both training and testing. It also enables cross-dataset training without any human interference.
  arXiv  Detail & Related papers  (2023-11-14T18:59:01Z)
• Weakly-supervised Contrastive Learning for Unsupervised Object Discovery [52.696041556640516]
  Unsupervised object discovery is promising due to its ability to discover objects in a generic manner. We design a semantic-guided self-supervised learning model to extract high-level semantic features from images. We introduce Principal Component Analysis (PCA) to localize object regions.
  arXiv  Detail & Related papers  (2023-07-07T04:03:48Z)
• Learning Open-World Object Proposals without Learning to Classify [110.30191531975804]
  We propose a classification-free Object Localization Network (OLN) which estimates the objectness of each region purely by how well the location and shape of a region overlaps with any ground-truth object. This simple strategy learns generalizable objectness and outperforms existing proposals on cross-category generalization.
  arXiv  Detail & Related papers  (2021-08-15T14:36:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.