Benchmarking Multi-View BEV Object Detection with Mixed Pinhole and Fisheye Cameras

• URL: http://arxiv.org/abs/2603.27818v1
• Date: Sun, 29 Mar 2026 19:10:34 GMT
• Title: Benchmarking Multi-View BEV Object Detection with Mixed Pinhole and Fisheye Cameras
• Abstract summary: We introduce a multi-view BEV detection benchmark with mixed cameras by converting KITTI-360 into nuScenes format.<n>Our study encompasses three adaptations: rectification for zero-shot evaluation and fine-tuning of nuScenes-trained models.<n>We demonstrate that projection-free architectures are inherently more robust and effective against fisheye distortion than other VTMs.
• Score: 5.227238848375275
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Modern autonomous driving systems increasingly rely on mixed camera configurations with pinhole and fisheye cameras for full view perception. However, Bird's-Eye View (BEV) 3D object detection models are predominantly designed for pinhole cameras, leading to performance degradation under fisheye distortion. To bridge this gap, we introduce a multi-view BEV detection benchmark with mixed cameras by converting KITTI-360 into nuScenes format. Our study encompasses three adaptations: rectification for zero-shot evaluation and fine-tuning of nuScenes-trained models, distortion-aware view transformation modules (VTMs) via the MEI camera model, and polar coordinate representations to better align with radial distortion. We systematically evaluate three representative BEV architectures, BEVFormer, BEVDet and PETR, across these strategies. We demonstrate that projection-free architectures are inherently more robust and effective against fisheye distortion than other VTMs. This work establishes the first real-data 3D detection benchmark with fisheye and pinhole images and provides systematic adaptation and practical guidelines for designing robust and cost-effective 3D perception systems. The code is available at https://github.com/CesarLiu/FishBEVOD.git.

Related papers

• OmniPoint: Universal Monocular Metric Pointcloud from Any Camera [79.24417597712802]
  We present OmniPoint, a unified framework designed to generalize metric reconstruction across diverse imaging sensors.<n>To overcome projection rigidity, OmniPoint abandons conventional planar depth regression.<n>It instead adopts a decoupled ray and distance representation alongside a decoupled training objective.
  arXiv  Detail & Related papers  (2026-09-08T19:45:58Z)
• Distortion-Aware PETR for BEV Object Detection with Mixed Pinhole-Fisheye Cameras [0.0]
  Fisheye cameras are widely deployed in autonomous driving perception suites for their low cost and full-coverage field of view (FOV)<n>Severe radial distortion challenges most BEV detectors by violating the fundamental assumption of uniform sampling.<n>We propose Distortion-Aware PETR (DAPETR), a projection-free detector tailored for mixed pinhole-fisheye camera setups.
  arXiv  Detail & Related papers  (2026-06-07T15:32:10Z)
• Fisheye3R: Adapting Unified 3D Feed-Forward Foundation Models to Fisheye Lenses [5.987299558427844]
  Fisheye3R is a novel adaptation framework that extends multi-view 3D reconstruction foundation models to accommodate fisheye inputs without performance regression on perspective images.<n>Our approach consistently improves camera pose, depth, point map, and field-of-view estimation on fisheye images.
  arXiv  Detail & Related papers  (2026-03-30T18:18:56Z)
• CAM3R: Camera-Agnostic Model for 3D Reconstruction [28.364643298648726]
  We present CAM3R, a Camera-Agnostic, feed-forward Model for 3D Reconstruction capable of processing images from wide-angle camera models without prior calibration.<n>Our framework consists of a two-view network which is bifurcated into a Ray Module and a Cross-view Module.<n>To unify these pairwise predictions into a consistent 3D scene, we introduce a Ray-Aware Global Alignment framework.
  arXiv  Detail & Related papers  (2026-03-23T23:20:15Z)
• UniK3D: Universal Camera Monocular 3D Estimation [62.06785782635153]
  We present UniK3D, the first generalizable method for monocular 3D estimation able to model any camera.<n>Our method introduces a spherical 3D representation which allows for better disentanglement of camera and scene geometry.<n>A comprehensive zero-shot evaluation on 13 diverse datasets demonstrates the state-of-the-art performance of UniK3D across 3D, depth, and camera metrics.
  arXiv  Detail & Related papers  (2025-03-20T17:49:23Z)
• SDGE: Stereo Guided Depth Estimation for 360$^\circ$ Camera Sets [65.64958606221069]
  Multi-camera systems are often used in autonomous driving to achieve a 360$circ$ perception. These 360$circ$ camera sets often have limited or low-quality overlap regions, making multi-view stereo methods infeasible for the entire image. We propose the Stereo Guided Depth Estimation (SGDE) method, which enhances depth estimation of the full image by explicitly utilizing multi-view stereo results on the overlap.
  arXiv  Detail & Related papers  (2024-02-19T02:41:37Z)
• Instance-aware Multi-Camera 3D Object Detection with Structural Priors Mining and Self-Boosting Learning [93.71280187657831]
  Camera-based bird-eye-view (BEV) perception paradigm has made significant progress in the autonomous driving field. We propose IA-BEV, which integrates image-plane instance awareness into the depth estimation process within a BEV-based detector.
  arXiv  Detail & Related papers  (2023-12-13T09:24:42Z)
• Towards Generalizable Multi-Camera 3D Object Detection via Perspective Debiasing [28.874014617259935]
  Multi-Camera 3D Object Detection (MC3D-Det) has gained prominence with the advent of bird's-eye view (BEV) approaches. We propose a novel method that aligns 3D detection with 2D camera plane results, ensuring consistent and accurate detections.
  arXiv  Detail & Related papers  (2023-10-17T15:31:28Z)
• Depth Estimation Analysis of Orthogonally Divergent Fisheye Cameras with Distortion Removal [0.0]
  Traditional stereo vision systems may not be suitable for certain scenarios due to their limited field of view. Fisheye cameras introduce significant distortion at the edges that affects the accuracy of stereo matching and depth estimation. This paper proposes a method for distortion-removal and depth estimation analysis for stereovision system.
  arXiv  Detail & Related papers  (2023-07-07T13:44:12Z)
• GeoMIM: Towards Better 3D Knowledge Transfer via Masked Image Modeling for Multi-view 3D Understanding [42.780417042750315]
  Multi-view camera-based 3D detection is a challenging problem in computer vision. Recent works leverage a pretrained LiDAR detection model to transfer knowledge to a camera-based student network. We propose Enhanced Geometry Masked Image Modeling (GeoMIM) to transfer the knowledge of the LiDAR model in a pretrain-finetune paradigm.
  arXiv  Detail & Related papers  (2023-03-20T17:59:03Z)
• Towards Model Generalization for Monocular 3D Object Detection [57.25828870799331]
  We present an effective unified camera-generalized paradigm (CGP) for Mono3D object detection. We also propose the 2D-3D geometry-consistent object scaling strategy (GCOS) to bridge the gap via an instance-level augment. Our method called DGMono3D achieves remarkable performance on all evaluated datasets and surpasses the SoTA unsupervised domain adaptation scheme.
  arXiv  Detail & Related papers  (2022-05-23T23:05:07Z)
• Wide-angle Image Rectification: A Survey [86.36118799330802]
  wide-angle images contain distortions that violate the assumptions underlying pinhole camera models. Image rectification, which aims to correct these distortions, can solve these problems. We present a detailed description and discussion of the camera models used in different approaches. Next, we review both traditional geometry-based image rectification methods and deep learning-based methods.
  arXiv  Detail & Related papers  (2020-10-30T17:28:40Z)
• Lightweight Multi-View 3D Pose Estimation through Camera-Disentangled Representation [57.11299763566534]
  We present a solution to recover 3D pose from multi-view images captured with spatially calibrated cameras. We exploit 3D geometry to fuse input images into a unified latent representation of pose, which is disentangled from camera view-points. Our architecture then conditions the learned representation on camera projection operators to produce accurate per-view 2d detections.
  arXiv  Detail & Related papers  (2020-04-05T12:52:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.