MVDGC: Joint 3D and 2D Multi-view Pedestrian Detection via Dual Geometric Constraints

• URL: http://arxiv.org/abs/2607.00273v1
• Date: Tue, 30 Jun 2026 23:39:38 GMT
• Title: MVDGC: Joint 3D and 2D Multi-view Pedestrian Detection via Dual Geometric Constraints
• Abstract summary: We propose a unified framework that estimates pedestrian locations on the BEV plane and 2D bounding boxes in image views.<n>MVDGC employs a emphsparse set of 3D cylindrical queries that embraces geometric context across both BEV and image views.<n>These queries function as shape anchors that directly extract 2D features from the intact image-view features using camera projection.
• Score: 11.583454260842524
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The core challenge in multi-view pedestrian detection (MVPD) lies in effective aggregation of visual features from different viewpoints for robust occlusion reasoning. Recent approaches have addressed this by first projecting image-view features onto a Bird's Eye View (BEV) map, where ground localization is then performed. Despite impressive performance, the perspective transformation induces severe distortion, causing spatial structure break and degrading the quality of object feature extraction. The blurred and ambiguous features hinder accurate BEV point localization, especially in densely populated regions. Moreover, the strong mutual relationship between the BEV ground point and image bounding boxes is not capitalized on. Although multi-view consistency of 2D detections can serve as a powerful constraint in BEV space, these detections are commonly treated as auxiliary signals rather than being jointly optimized with the primary task.In this work, we propose \textbf{MVDGC}, a unified framework that \emph{jointly estimates pedestrian locations on the BEV plane and 2D bounding boxes in image views}. MVDGC employs a \emph{sparse set of 3D cylindrical queries} that embraces geometric context across both BEV and image views, enforcing dual spatial constraints for precise localization. Specifically, the geometric constraints is established by modeling each pedestrian as a vertical cylinder whose center lies on the BEV plane and whose projection casts a rectangular box in the image views. These queries function as shape anchors that directly extract 2D features from the intact image-view features using camera projection, eliminating projection-induced distortions. The 3D cylindrical query enables the unification of BEV and ImV localization into a single task: 3D cylinder position and shape refinement. Code is available at: https://github.com/UARK-AICV/MVDGC

Related papers

• Open-Vocabulary BEV Segmentation with 3D-Aware Geometric Constraints [20.26519299938903]
  We introduce open-vocabulary BEV segmentation (OVBS) to recognize categories beyond the training set.<n>OVBEVSeg is a geometry-aware OVBS framework that enhances efficient Gaussian splatting (GS)-based unprojection.<n>On the nuScenes dataset, OVBEVSeg achieves state-of-the-art performance, outperforming closed-set methods by 15.3 mIoU on unseen categories.
  arXiv  Detail & Related papers  (2026-06-23T09:43:12Z)
• Unifying UAV Cross-View Geo-Localization via 3D Geometric Perception [51.687842983240564]
  Cross-view geo-localization for Unmanned Aerial Vehicles (UAVs) remains challenging due to the severe geometric discrepancy between oblique UAV imagery and satellite maps.<n>We propose a geometry-aware UAV geo-localization framework that explicitly models the 3D scene geometry to unify coarse place recognition and fine-grained pose estimation.<n>Our method significantly outperforms state-of-the-art baselines, achieving robust meter-level localization accuracy and improved generalization in complex urban environments.
  arXiv  Detail & Related papers  (2026-04-02T08:08:41Z)
• SpatialFly: Geometry-Guided Representation Alignment for UAV Vision-and-Language Navigation in Urban Environments [49.966170814478915]
  We propose a geometry-guided spatial representation framework for UAV VLN.<n>Running on RGB observations without explicit 3D reconstruction, SpatialFly introduces a geometry-guided 2D representation alignment mechanism.<n> Experimental results show that SpatialFly consistently outperforms state-of-the-art UAV VLN baselines across both seen and unseen environments.
  arXiv  Detail & Related papers  (2026-03-22T03:56:58Z)
• Reconstruction Matters: Learning Geometry-Aligned BEV Representation through 3D Gaussian Splatting [24.49684091836623]
  Splat2BEV aims to learn BEV feature representations that are both semantically rich and geometrically precise.<n>Experiments on nuScenes and argoverse dataset demonstrate that Splat2BEV achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-03-19T17:49:43Z)
• GeoBEV: Learning Geometric BEV Representation for Multi-view 3D Object Detection [36.245654685143016]
  Bird's-Eye-View (BEV) representation has emerged as a mainstream paradigm for multi-view 3D object detection.<n>Existing methods overlook the geometric quality of BEV representation, leaving it in a low-resolution state.<n>We propose Radial-Cartesian BEV Sampling (RC-Sampling) to generate high-resolution dense BEV representation.<n>In conjunction with the In-Box Label, Centroid-Aware Inner Loss (CAI Loss) is developed to capture the inner geometric structure of objects.
  arXiv  Detail & Related papers  (2024-09-03T11:57:36Z)
• BEV-IO: Enhancing Bird's-Eye-View 3D Detection with Instance Occupancy [58.92659367605442]
  We present BEV-IO, a new 3D detection paradigm to enhance BEV representation with instance occupancy information. We show that BEV-IO can outperform state-of-the-art methods while only adding a negligible increase in parameters and computational overhead.
  arXiv  Detail & Related papers  (2023-05-26T11:16:12Z)
• Geometric-aware Pretraining for Vision-centric 3D Object Detection [77.7979088689944]
  We propose a novel geometric-aware pretraining framework called GAPretrain. GAPretrain serves as a plug-and-play solution that can be flexibly applied to multiple state-of-the-art detectors. We achieve 46.2 mAP and 55.5 NDS on the nuScenes val set using the BEVFormer method, with a gain of 2.7 and 2.1 points, respectively.
  arXiv  Detail & Related papers  (2023-04-06T14:33:05Z)
• FrustumFormer: Adaptive Instance-aware Resampling for Multi-view 3D Detection [47.6570523164125]
  We propose a novel framework named FrustumFormer, which pays more attention to the features in instance regions via adaptive instance-aware resampling. Experiments on the nuScenes dataset demonstrate the effectiveness of FrustumFormer, and we achieve a new state-of-the-art performance on the benchmark.
  arXiv  Detail & Related papers  (2023-01-10T17:51:55Z)
• GitNet: Geometric Prior-based Transformation for Birds-Eye-View Segmentation [105.19949897812494]
  Birds-eye-view (BEV) semantic segmentation is critical for autonomous driving. We present a novel two-stage Geometry Prior-based Transformation framework named GitNet.
  arXiv  Detail & Related papers  (2022-04-16T06:46:45Z)
• M^2BEV: Multi-Camera Joint 3D Detection and Segmentation with Unified Birds-Eye View Representation [145.6041893646006]
  M$2$BEV is a unified framework that jointly performs 3D object detection and map segmentation. M$2$BEV infers both tasks with a unified model and improves efficiency.
  arXiv  Detail & Related papers  (2022-04-11T13:43:25Z)
• Bird's-Eye-View Panoptic Segmentation Using Monocular Frontal View Images [4.449481309681663]
  We present the first end-to-end learning approach for directly predicting dense panoptic segmentation maps in the Bird's-Eye-View (BEV) maps. Our architecture follows the top-down paradigm and incorporates a novel dense transformer module. We derive a mathematical formulation for the sensitivity of the FV-BEV transformation which allows us to intelligently weight pixels in the BEV space.
  arXiv  Detail & Related papers  (2021-08-06T17:59:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.