Leveraging Previous-Traversal Point Cloud Map Priors for Camera-Based 3D Object Detection and Tracking

• URL: http://arxiv.org/abs/2604.25405v1
• Date: Tue, 28 Apr 2026 09:16:44 GMT
• Title: Leveraging Previous-Traversal Point Cloud Map Priors for Camera-Based 3D Object Detection and Tracking
• Abstract summary: We propose DualViewMapDet, a camera-only inference framework that retrieves map priors online.<n>The key idea is a dual-space camera-map fusion strategy that avoids one-sided view conversion.<n>We make the code and pre-trained models available at https://dualviewmapdet.cs.uni-freiburg.de.
• Score: 15.914966195454403
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Camera-based 3D object detection and tracking are central to autonomous driving, yet precise 3D object localization remains fundamentally constrained by depth ambiguity when no expensive, depth-rich online LiDAR is available at inference. In many deployments, however, vehicles repeatedly traverse the same environments, making static point cloud maps from prior traversals a practical source of geometric priors. We propose DualViewMapDet, a camera-only inference framework that retrieves such map priors online and leverages them to mitigate the absence of a LiDAR sensor during deployment. The key idea is a dual-space camera-map fusion strategy that avoids one-sided view conversion. Specifically, we (i) project the map into perspective view (PV) and encode multi-channel geometric cues to enrich image features and support BEV lifting, and (ii) encode the map directly in bird's-eye view (BEV) with a sparse voxel backbone and fuse it with lifted camera features in a shared metric space. Extensive evaluations on nuScenes and Argoverse 2 demonstrate consistent improvements over strong camera-only baselines, with particularly strong gains in object localization. Ablations further validate the contributions of PV/BEV fusion and prior-map coverage. We make the code and pre-trained models available at https://dualviewmapdet.cs.uni-freiburg.de .

Related papers

• Map-Det3D: Metric Feed-Forward 3D Reconstruction Prior for Multi-view 3D Object Detection from Streaming Inputs [62.865932175477035]
  Metric 3D object detection is a core capability for embodied agents, yet most reliable systems lean on depth sensors.<n>We propose Map-Det3D, an online multi-view 3D object detection model that brings detection directly into a 3D space reconstructed from RGB.
  arXiv  Detail & Related papers  (2026-08-12T15:33:42Z)
• S3KF: Spherical State-Space Kalman Filtering for Panoramic 3D Multi-Object Tracking [24.79542959759503]
  Panoramic multi-object tracking is important for industrial safety monitoring, wide-area robotic perception, and infrastructure-light deployment in large workspaces.<n>Existing image-plane trackers are tightly coupled to the camera projection and become unreliable in panoramic imagery.<n>We present a panoramic 3D multi-object tracking framework built on a motorized rotating LiDAR and a quad-fisheye camera rig.
  arXiv  Detail & Related papers  (2026-03-29T06:01:29Z)
• VGGT: Visual Geometry Grounded Transformer [61.37669770946458]
  VGGT is a feed-forward neural network that directly infers all key 3D attributes of a scene.<n>Network achieves state-of-the-art results in multiple 3D tasks.
  arXiv  Detail & Related papers  (2025-03-14T17:59:47Z)
• Towards Generalizable Multi-Camera 3D Object Detection via Perspective Debiasing [28.874014617259935]
  Multi-Camera 3D Object Detection (MC3D-Det) has gained prominence with the advent of bird's-eye view (BEV) approaches. We propose a novel method that aligns 3D detection with 2D camera plane results, ensuring consistent and accurate detections.
  arXiv  Detail & Related papers  (2023-10-17T15:31:28Z)
• CoBEV: Elevating Roadside 3D Object Detection with Depth and Height Complementarity [34.025530326420146]
  We develop Complementary-BEV, a novel end-to-end monocular 3D object detection framework. We conduct extensive experiments on the public 3D detection benchmarks of roadside camera-based DAIR-V2X-I and Rope3D. For the first time, the vehicle AP score of a camera model reaches 80% on DAIR-V2X-I in terms of easy mode.
  arXiv  Detail & Related papers  (2023-10-04T13:38:53Z)
• Multi-camera Bird's Eye View Perception for Autonomous Driving [17.834495597639805]
  It is essential to produce perception outputs in 3D to enable the spatial reasoning of other agents and structures. The most basic approach to achieving the desired BEV representation from a camera image is IPM, assuming a flat ground surface. More recent approaches use deep neural networks to output directly in BEV space.
  arXiv  Detail & Related papers  (2023-09-16T19:12:05Z)
• UniScene: Multi-Camera Unified Pre-training via 3D Scene Reconstruction for Autonomous Driving [11.507979392707448]
  We propose the first multi-camera unified pre-training framework, called UniScene. We employ Occupancy as the general representation for the 3D scene, enabling the model to grasp geometric priors of the surrounding world. UniScene shows a significant improvement of about 2.0% in mAP and 2.0% in NDS for multi-camera 3D object detection, as well as a 3% increase in mIoU for surrounding semantic scene completion.
  arXiv  Detail & Related papers  (2023-05-30T08:23:06Z)
• Geometric-aware Pretraining for Vision-centric 3D Object Detection [77.7979088689944]
  We propose a novel geometric-aware pretraining framework called GAPretrain. GAPretrain serves as a plug-and-play solution that can be flexibly applied to multiple state-of-the-art detectors. We achieve 46.2 mAP and 55.5 NDS on the nuScenes val set using the BEVFormer method, with a gain of 2.7 and 2.1 points, respectively.
  arXiv  Detail & Related papers  (2023-04-06T14:33:05Z)
• GeoMIM: Towards Better 3D Knowledge Transfer via Masked Image Modeling for Multi-view 3D Understanding [42.780417042750315]
  Multi-view camera-based 3D detection is a challenging problem in computer vision. Recent works leverage a pretrained LiDAR detection model to transfer knowledge to a camera-based student network. We propose Enhanced Geometry Masked Image Modeling (GeoMIM) to transfer the knowledge of the LiDAR model in a pretrain-finetune paradigm.
  arXiv  Detail & Related papers  (2023-03-20T17:59:03Z)
• BEV-MAE: Bird's Eye View Masked Autoencoders for Point Cloud Pre-training in Autonomous Driving Scenarios [51.285561119993105]
  We present BEV-MAE, an efficient masked autoencoder pre-training framework for LiDAR-based 3D object detection in autonomous driving. Specifically, we propose a bird's eye view (BEV) guided masking strategy to guide the 3D encoder learning feature representation. We introduce a learnable point token to maintain a consistent receptive field size of the 3D encoder.
  arXiv  Detail & Related papers  (2022-12-12T08:15:03Z)
• Satellite Image Based Cross-view Localization for Autonomous Vehicle [59.72040418584396]
  This paper shows that by using an off-the-shelf high-definition satellite image as a ready-to-use map, we are able to achieve cross-view vehicle localization up to a satisfactory accuracy. Our method is validated on KITTI and Ford Multi-AV Seasonal datasets as ground view and Google Maps as the satellite view.
  arXiv  Detail & Related papers  (2022-07-27T13:16:39Z)
• BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation [105.96557764248846]
  We introduce BEVFusion, a generic multi-task multi-sensor fusion framework. It unifies multi-modal features in the shared bird's-eye view representation space. It achieves 1.3% higher mAP and NDS on 3D object detection and 13.6% higher mIoU on BEV map segmentation, with 1.9x lower cost.
  arXiv  Detail & Related papers  (2022-05-26T17:59:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.