Detect Anything in Real Time: From Single-Prompt Segmentation to Multi-Class Detection

• URL: http://arxiv.org/abs/2603.11441v1
• Date: Thu, 12 Mar 2026 02:05:11 GMT
• Title: Detect Anything in Real Time: From Single-Prompt Segmentation to Multi-Class Detection
• Abstract summary: We present a training-free framework that converts SAM3 into a real-time multi-class detector by exploiting a structural invariant.<n>On val 2017 (5,000 images, 80 classes), DART achieves 55.8 AP at 15.8 FPS (4 classes, 1008x1008) on a single GTX 4080.<n>For extreme latency targets, adapter distillation with a frozen encoder-decoder achieves 38.7 AP with a 13.9 ms backbone.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in vision-language modeling have produced promptable detection and segmentation systems that accept arbitrary natural language queries at inference time. Among these, SAM3 achieves state-of-the-art accuracy by combining a ViT-H/14 backbone with cross-modal transformer decoding and learned object queries. However, SAM3 processes a single text prompt per forward pass. Detecting N categories requires N independent executions, each dominated by the 439M-parameter backbone. We present Detect Anything in Real Time (DART), a training-free framework that converts SAM3 into a real-time multi-class detector by exploiting a structural invariant: the visual backbone is class-agnostic, producing image features independent of the text prompt. This allows the backbone computation to be shared between all classes, reducing its cost from O(N) to O(1). Combined with batched multi-class decoding, detection-only inference, and TensorRT FP16 deployment, these optimizations yield 5.6x cumulative speedup at 3 classes, scaling to 25x at 80 classes, without modifying any model weight. On COCO val2017 (5,000 images, 80 classes), DART achieves 55.8 AP at 15.8 FPS (4 classes, 1008x1008) on a single RTX 4080, surpassing purpose-built open-vocabulary detectors trained on millions of box annotations. For extreme latency targets, adapter distillation with a frozen encoder-decoder achieves 38.7 AP with a 13.9 ms backbone. Code and models are available at https://github.com/mkturkcan/DART.

Related papers

• ISE: An Execution-Grounded Recipe for Multi-Turn OS-Agent Trajectories [18.739245125968175]
  Training capable OS agents requires data that simultaneously captures structured user intents, multi-turn task delegation, and grounded tool execution--properties absent from existing datasets.<n>We propose ISE (Intent -> Simulate -> Execute), a three-stage synthesis paradigm that addresses these gaps jointly.
  arXiv  Detail & Related papers  (2026-06-09T23:44:26Z)
• CMAP: Cross-Modal Adaptive Prompting for Multi-Domain Task-Incremental Learning [3.1368611610608856]
  Text-space task routing replaces visual Gaussian matching with cosine similarity to frozen CLIP text prototypes.<n>Visual-textual confidence replaces single-Gaussian class modeling with K-means visual prototypes and cross-modal alignment scores under task-calibrated thresholds.<n>On the MTIL benchmark spanning 118.7% and 1201 classes, our method achieves 74.2% Transfer, 80.5% Average, and 8 Last under Order-I.
  arXiv  Detail & Related papers  (2026-05-25T11:09:48Z)
• OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models [69.2503510410147]
  We present a unified autonomous driving framework built upon a pretrained VLM.<n>We demonstrate that pretrained VLM attention exhibits strong transferability beyond pure language modeling.<n>Experiments on end-to-end autonomous driving benchmarks demonstrate state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-04-20T07:50:00Z)
• COVTrack++: Learning Open-Vocabulary Multi-Object Tracking from Continuous Videos via a Synergistic Paradigm [59.26203051651017]
  C-TAO is the first continuously annotated training set for Open-Vocabulary Multi-Object Tracking (OVMOT)<n>For the framework bottleneck, we propose COVTrack++, a synergistic framework that achieves a bidirectional reciprocal mechanism between detection and association through three modules.<n>Experiments on TAO demonstrate state-of-the-art performance, with novel TETA reaching 35.4% and 30.5% on validation and test sets, improving novel AssocA by 4.8% and novel LocA by 5.8% over previous methods.
  arXiv  Detail & Related papers  (2026-03-25T07:20:27Z)
• The Mirror Design Pattern: Strict Data Geometry over Model Scale for Prompt Injection Detection [0.0]
  We introduce Mirror, a data-curation design pattern that organizes prompt injection corpora into matched positive and negative cells.<n>We show that for L1 prompt injection screening, strict data geometry can matter more than model scale.
  arXiv  Detail & Related papers  (2026-03-12T12:50:57Z)
• dVoting: Fast Voting for dLLMs [71.572316901001]
  Diffusion Large Language Models (dLLMs) represent a new paradigm beyond autoregressive modeling.<n>dLLMs can generate tokens at arbitrary positions in parallel, endowing them with significant potential for parallel test-time scaling.<n>We introduce dVoting, a fast voting technique that boosts reasoning capability without training, with only an acceptable extra computational overhead.
  arXiv  Detail & Related papers  (2026-02-12T16:35:05Z)
• Lightweight LiDAR-Camera 3D Dynamic Object Detection and Multi-Class Trajectory Prediction [7.415417400188903]
  Service mobile robots are often required to avoid dynamic objects while performing their tasks.<n>We present a lightweight multi-modal framework for 3D object detection and trajectory prediction.<n>Our system integrates LiDAR and camera inputs to achieve real-time perception of pedestrians, vehicles, and riders in 3D space.
  arXiv  Detail & Related papers  (2025-04-18T11:59:34Z)
• S*: Test Time Scaling for Code Generation [55.11863577956177]
  We propose S*, the first hybrid test-time scaling framework for code generation.<n>S* substantially improves the coverage and selection accuracy of generated code.
  arXiv  Detail & Related papers  (2025-02-20T09:18:53Z)
• RT-OVAD: Real-Time Open-Vocabulary Aerial Object Detection via Image-Text Collaboration [12.66046875297631]
  We propose RT-OVAD, the first real-time open-vocabulary detector for aerial scenes.<n>We introduce an image-to-text alignment loss to replace the conventional category regression loss.<n>We also propose a lightweight image-text collaboration strategy comprising an image-text collaboration encoder and a text-guided decoder.
  arXiv  Detail & Related papers  (2024-08-22T09:33:25Z)
• Towards Robust Real-Time Scene Text Detection: From Semantic to Instance Representation Learning [19.856492291263102]
  We propose representation learning for real-time scene text detection. For semantic representation learning, we propose global-dense semantic contrast (GDSC) and top-down modeling (TDM) With the proposed GDSC and TDM, the encoder network learns stronger representation without introducing any parameters and computations during inference. The proposed method achieves 87.2% F-measure with 48.2 FPS on Total-Text and 89.6% F-measure with 36.9 FPS on MSRA-TD500.
  arXiv  Detail & Related papers  (2023-08-14T15:14:37Z)
• Semantic-Aligned Matching for Enhanced DETR Convergence and Multi-Scale Feature Fusion [95.7732308775325]
  The proposed DEtection TRansformer (DETR) has established a fully end-to-end paradigm for object detection. DETR suffers from slow training convergence, which hinders its applicability to various detection tasks. We design Semantic-Aligned-Matching DETR++ to accelerate DETR's convergence and improve detection performance.
  arXiv  Detail & Related papers  (2022-07-28T15:34:29Z)
• MSANet: Multi-Similarity and Attention Guidance for Boosting Few-Shot Segmentation [0.0]
  Few-shot segmentation aims to segment unseen-class objects given only a handful of densely labeled samples. Prototype learning, where the support feature yields a singleor several prototypes, has been widely used in FSS. We propose a Multi-Similarity and Attention Network (MSANet) including two novel modules, a multi-similarity module and an attention module.
  arXiv  Detail & Related papers  (2022-06-20T09:14:17Z)
• EAutoDet: Efficient Architecture Search for Object Detection [110.99532343155073]
  EAutoDet framework can discover practical backbone and FPN architectures for object detection in 1.4 GPU-days. We propose a kernel reusing technique by sharing the weights of candidate operations on one edge and consolidating them into one convolution. In particular, the discovered architectures surpass state-of-the-art object detection NAS methods and achieve 40.1 mAP with 120 FPS and 49.2 mAP with 41.3 FPS on COCO test-dev set.
  arXiv  Detail & Related papers  (2022-03-21T05:56:12Z)
• UniFormer: Unifying Convolution and Self-attention for Visual Recognition [69.68907941116127]
  Convolution neural networks (CNNs) and vision transformers (ViTs) have been two dominant frameworks in the past few years. We propose a novel Unified transFormer (UniFormer) which seamlessly integrates the merits of convolution and self-attention in a concise transformer format. Our UniFormer achieves 86.3 top-1 accuracy on ImageNet-1K classification.
  arXiv  Detail & Related papers  (2022-01-24T04:39:39Z)
• Tracking by Instance Detection: A Meta-Learning Approach [99.66119903655711]
  We propose a principled three-step approach to build a high-performance tracker. We build two trackers, named Retina-MAML and FCOS-MAML, based on two modern detectors RetinaNet and FCOS. Both trackers run in real-time at 40 FPS.
  arXiv  Detail & Related papers  (2020-04-02T05:55:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.