Grounding DINO 1.5: Advance the "Edge" of Open-Set Object Detection

• URL: http://arxiv.org/abs/2405.10300v2
• Date: Sat, 1 Jun 2024 03:35:22 GMT
• Title: Grounding DINO 1.5: Advance the "Edge" of Open-Set Object Detection
• Authors: Tianhe Ren, Qing Jiang, Shilong Liu, Zhaoyang Zeng, Wenlong Liu, Han Gao, Hongjie Huang, Zhengyu Ma, Xiaoke Jiang, Yihao Chen, Yuda Xiong, Hao Zhang, Feng Li, Peijun Tang, Kent Yu, Lei Zhang,
• Abstract summary: Grounding DINO 1.5 is a suite of advanced open-set object detection models developed by IDEA Research. Grounding DINO 1.5 Pro is a high-performance model designed for stronger generalization capability across a wide range of scenarios. Grounding DINO 1.5 Edge is an efficient optimized model for faster speed demanded in many applications requiring edge deployment.
• Score: 23.464027681439706
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper introduces Grounding DINO 1.5, a suite of advanced open-set object detection models developed by IDEA Research, which aims to advance the "Edge" of open-set object detection. The suite encompasses two models: Grounding DINO 1.5 Pro, a high-performance model designed for stronger generalization capability across a wide range of scenarios, and Grounding DINO 1.5 Edge, an efficient model optimized for faster speed demanded in many applications requiring edge deployment. The Grounding DINO 1.5 Pro model advances its predecessor by scaling up the model architecture, integrating an enhanced vision backbone, and expanding the training dataset to over 20 million images with grounding annotations, thereby achieving a richer semantic understanding. The Grounding DINO 1.5 Edge model, while designed for efficiency with reduced feature scales, maintains robust detection capabilities by being trained on the same comprehensive dataset. Empirical results demonstrate the effectiveness of Grounding DINO 1.5, with the Grounding DINO 1.5 Pro model attaining a 54.3 AP on the COCO detection benchmark and a 55.7 AP on the LVIS-minival zero-shot transfer benchmark, setting new records for open-set object detection. Furthermore, the Grounding DINO 1.5 Edge model, when optimized with TensorRT, achieves a speed of 75.2 FPS while attaining a zero-shot performance of 36.2 AP on the LVIS-minival benchmark, making it more suitable for edge computing scenarios. Model examples and demos with API will be released at https://github.com/IDEA-Research/Grounding-DINO-1.5-API

Related papers

• A Light Perspective for 3D Object Detection [46.23578780480946]
  This paper introduces a novel approach that incorporates cutting-edge Deep Learning techniques into the feature extraction process. Our model, NextBEV, surpasses established feature extractors like ResNet50 and MobileNetV3. By fusing these lightweight proposals, we have enhanced the accuracy of the VoxelNet-based model by 2.93% and improved the F1-score of the PointPillar-based model by approximately 20%.
  arXiv  Detail & Related papers  (2025-03-10T10:03:23Z)
• EMOv2: Pushing 5M Vision Model Frontier [92.21687467702972]
  We set up the new frontier of the 5M magnitude lightweight model on various downstream tasks. Our work rethinks the lightweight infrastructure of efficient IRB and practical components in Transformer. Considering the imperceptible latency for mobile users when downloading models under 4G/5G bandwidth, we investigate the performance upper limit of lightweight models with a magnitude of 5M.
  arXiv  Detail & Related papers  (2024-12-09T17:12:22Z)
• DINO-X: A Unified Vision Model for Open-World Object Detection and Understanding [35.522774800394664]
  We introduce DINO-X, which is a unified object-centric vision model developed by IDEA Research. DINO-X employs the same Transformer-based encoder-decoder architecture as Grounding DINO 1.5 to pursue an object-level representation for open-world object understanding. We develop a universal object prompt to support prompt-free open-world detection, making it possible to detect anything in an image without requiring users to provide any prompt.
  arXiv  Detail & Related papers  (2024-11-21T17:42:20Z)
• From Blurry to Brilliant Detection: YOLOv5-Based Aerial Object Detection with Super Resolution [4.107182710549721]
  We present an innovative approach that combines super-resolution and an adapted lightweight YOLOv5 architecture. Our experimental results demonstrate the model's superior performance in detecting small and densely clustered objects.
  arXiv  Detail & Related papers  (2024-01-26T05:50:58Z)
• Semantic Segmentation in Satellite Hyperspectral Imagery by Deep Learning [54.094272065609815]
  We propose a lightweight 1D-CNN model, 1D-Justo-LiuNet, which outperforms state-of-the-art models in the hypespectral domain. 1D-Justo-LiuNet achieves the highest accuracy (0.93) with the smallest model size (4,563 parameters) among all tested models.
  arXiv  Detail & Related papers  (2023-10-24T21:57:59Z)
• Ultra-low Power Deep Learning-based Monocular Relative Localization Onboard Nano-quadrotors [64.68349896377629]
  This work presents a novel autonomous end-to-end system that addresses the monocular relative localization, through deep neural networks (DNNs), of two peer nano-drones. To cope with the ultra-constrained nano-drone platform, we propose a vertically-integrated framework, including dataset augmentation, quantization, and system optimizations. Experimental results show that our DNN can precisely localize a 10cm-size target nano-drone by employing only low-resolution monochrome images, up to 2m distance.
  arXiv  Detail & Related papers  (2023-03-03T14:14:08Z)
• A Light-weight Deep Learning Model for Remote Sensing Image Classification [70.66164876551674]
  We present a high-performance and light-weight deep learning model for Remote Sensing Image Classification (RSIC) By conducting extensive experiments on the NWPU-RESISC45 benchmark, our proposed teacher-student models outperforms the state-of-the-art systems.
  arXiv  Detail & Related papers  (2023-02-25T09:02:01Z)
• DEYO: DETR with YOLO for Step-by-Step Object Detection [0.0]
  This paper proposes a new two-stage object detection model, named DETR with YOLO (DEYO) The first stage provides high-quality query and anchor feeding into the second stage, improving the performance and efficiency of the second stage compared to the original DETR model. Experiments demonstrate that DEYO attains 50.6 AP and 52.1 AP in 12 and 36 epochs, respectively.
  arXiv  Detail & Related papers  (2022-11-12T06:36:17Z)
• Optimizing Anchor-based Detectors for Autonomous Driving Scenes [22.946814647030667]
  This paper summarizes model improvements and inference-time optimizations for the popular anchor-based detectors in autonomous driving scenes. Based on the high-performing RCNN-RS and RetinaNet-RS detection frameworks, we study a set of framework improvements to adapt the detectors to better detect small objects in crowd scenes.
  arXiv  Detail & Related papers  (2022-08-11T22:44:59Z)
• EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applications [68.35683849098105]
  We introduce split depth-wise transpose attention (SDTA) encoder that splits input tensors into multiple channel groups. Our EdgeNeXt model with 1.3M parameters achieves 71.2% top-1 accuracy on ImageNet-1K. Our EdgeNeXt model with 5.6M parameters achieves 79.4% top-1 accuracy on ImageNet-1K.
  arXiv  Detail & Related papers  (2022-06-21T17:59:56Z)
• Disentangle Your Dense Object Detector [82.22771433419727]
  Deep learning-based dense object detectors have achieved great success in the past few years and have been applied to numerous multimedia applications such as video understanding. However, the current training pipeline for dense detectors is compromised to lots of conjunctions that may not hold. We propose Disentangled Dense Object Detector (DDOD), in which simple and effective disentanglement mechanisms are designed and integrated into the current state-of-the-art detectors.
  arXiv  Detail & Related papers  (2021-07-07T00:52:16Z)
• ANNETTE: Accurate Neural Network Execution Time Estimation with Stacked Models [56.21470608621633]
  We propose a time estimation framework to decouple the architectural search from the target hardware. The proposed methodology extracts a set of models from micro- kernel and multi-layer benchmarks and generates a stacked model for mapping and network execution time estimation. We compare estimation accuracy and fidelity of the generated mixed models, statistical models with the roofline model, and a refined roofline model for evaluation.
  arXiv  Detail & Related papers  (2021-05-07T11:39:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.