SimPB++: Simultaneously Detecting 2D and 3D Objects from Multiple Cameras

• URL: http://arxiv.org/abs/2605.01924v1
• Date: Sun, 03 May 2026 15:10:38 GMT
• Title: SimPB++: Simultaneously Detecting 2D and 3D Objects from Multiple Cameras
• Abstract summary: We propose SimPB++, which simultaneously detects 2D objects in perspective and 3D objects in Bird's Eye View (BEV) from multiple cameras.<n>It unifies both tasks into an end-to-end model with a hybrid decoder architecture, coupling multi-view 2D and 3D decoders interactively.<n> Experiments show state-of-the-art performance on nuScenes for both tasks and strong long-range detection (up to 150m) on Argoverse2.
• Score: 45.11375888769007
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Simultaneous perception of 2D objects in perspective view and 3D objects in Bird's Eye View (BEV) is challenging for multi-camera autonomous driving. Existing two-stage pipelines use 2D results only as a one-time cue for 3D detection. We propose SimPB++, which simultaneously detects 2D objects in perspective and 3D objects in BEV from multiple cameras. It unifies both tasks into an end-to-end model with a hybrid decoder architecture, coupling multi-view 2D and 3D decoders interactively. Two novel modules enable deep interaction: Dynamic Query Allocation adaptively assigns 2D queries to 3D candidates, and Adaptive Query Aggregation refines 3D representations using multi-view 2D features, forming a cyclic 3D-2D-3D refinement. For multi-view 2D detection, we use Query-group Attention for intra-group communication. We also design a Crop-and-Scale strategy for long-range perception and a Propagating Denoising strategy with an auxiliary RoI detector. SimPB++ supports mixed supervision with 2D-only and fully annotated data, reducing reliance on expensive 3D labels. Experiments show state-of-the-art performance on nuScenes for both tasks and strong long-range detection (up to 150m) on Argoverse2.

Related papers

• Map-Det3D: Metric Feed-Forward 3D Reconstruction Prior for Multi-view 3D Object Detection from Streaming Inputs [62.865932175477035]
  Metric 3D object detection is a core capability for embodied agents, yet most reliable systems lean on depth sensors.<n>We propose Map-Det3D, an online multi-view 3D object detection model that brings detection directly into a 3D space reconstructed from RGB.
  arXiv  Detail & Related papers  (2026-08-12T15:33:42Z)
• VR3D: View-Robust 3D Representation Learning for Aerial-Ground Person Re-Identification [35.655590576867574]
  Aerial-ground person re-identification is a challenging task due to cross-platform viewpoint variations.<n>Existing methods attempt to learn view-invariant representations exclusively within the 2D image space.<n>We propose VR3D, a View-Robust 3D Representation Learning framework that maps images into a unified 3D coordinate space.
  arXiv  Detail & Related papers  (2026-08-03T17:59:01Z)
• SegDINO3D: 3D Instance Segmentation Empowered by Both Image-Level and Object-Level 2D Features [61.13570953713554]
  SegDINO3D is a novel Transformer encoder-decoder framework for 3D instance segmentation.<n>It fully leverage 2D representation from a pre-trained 2D detection model, including both image-level and object-level features.<n>SegDINO3D achieves the state-of-the-art performance on the ScanNetV2 and ScanNet200 3D instance segmentation benchmarks.
  arXiv  Detail & Related papers  (2025-09-19T15:41:10Z)
• 3D-MOOD: Lifting 2D to 3D for Monocular Open-Set Object Detection [62.57179069154312]
  We introduce the first end-to-end 3D Monocular Open-set Object Detector (3D-MOOD)<n>We lift the open-set 2D detection into 3D space through our designed 3D bounding box head.<n>We condition the object queries with geometry prior and overcome the generalization for 3D estimation across diverse scenes.
  arXiv  Detail & Related papers  (2025-07-31T13:56:41Z)
• Roadside Monocular 3D Detection Prompted by 2D Detection [13.085987471715725]
  We introduce Promptable 3D Detector (Pro3D), a novel detector design that leverages 2D detections as prompts.<n>Pro3D is adaptable for use with a wide range of 2D and 3D detectors with minimal modifications.
  arXiv  Detail & Related papers  (2024-04-01T11:57:34Z)
• SimPB: A Single Model for 2D and 3D Object Detection from Multiple Cameras [3.648972014796591]
  We present a single model termed SimPB, which simultaneously detects 2D objects in the perspective view and 3D objects in the BEV space from multiple cameras. A hybrid decoder consists of several multi-view 2D decoder layers and several 3D decoder layers, specifically designed for their respective detection tasks.
  arXiv  Detail & Related papers  (2024-03-15T14:39:39Z)
• Enhancing 3D Object Detection with 2D Detection-Guided Query Anchors [6.3557174349423455]
  We present a novel query generating approach termed QAF2D, which infers 3D query anchors from 2D detection results. The largest improvement that QAF2D can bring about on the nuScenes validation subset is $2.3%$ NDS and $2.7%$ mAP.
  arXiv  Detail & Related papers  (2024-03-10T04:38:27Z)
• Tracking Objects with 3D Representation from Videos [57.641129788552675]
  We propose a new 2D Multiple Object Tracking paradigm, called P3DTrack. With 3D object representation learning from Pseudo 3D object labels in monocular videos, we propose a new 2D MOT paradigm, called P3DTrack.
  arXiv  Detail & Related papers  (2023-06-08T17:58:45Z)
• FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection [78.00922683083776]
  It is non-trivial to make a general adapted 2D detector work in this 3D task. In this technical report, we study this problem with a practice built on fully convolutional single-stage detector. Our solution achieves 1st place out of all the vision-only methods in the nuScenes 3D detection challenge of NeurIPS 2020.
  arXiv  Detail & Related papers  (2021-04-22T09:35:35Z)
• YOLOStereo3D: A Step Back to 2D for Efficient Stereo 3D Detection [6.5702792909006735]
  YOLOStereo3D is trained on one single GPU and runs at more than ten fps. It demonstrates performance comparable to state-of-the-art stereo 3D detection frameworks without usage of LiDAR data.
  arXiv  Detail & Related papers  (2021-03-17T03:43:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.