Distortion-Aware PETR for BEV Object Detection with Mixed Pinhole-Fisheye Cameras

• URL: http://arxiv.org/abs/2606.08680v1
• Date: Sun, 07 Jun 2026 15:32:10 GMT
• Title: Distortion-Aware PETR for BEV Object Detection with Mixed Pinhole-Fisheye Cameras
• Abstract summary: Fisheye cameras are widely deployed in autonomous driving perception suites for their low cost and full-coverage field of view (FOV)<n>Severe radial distortion challenges most BEV detectors by violating the fundamental assumption of uniform sampling.<n>We propose Distortion-Aware PETR (DAPETR), a projection-free detector tailored for mixed pinhole-fisheye camera setups.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fisheye cameras are widely deployed in autonomous driving perception suites for their low cost and full-coverage field of view (FOV), yet their potential remains underleveraged in 3D object detection. Severe radial distortion challenges most BEV detectors by violating the fundamental assumption of uniform sampling. To bridge this gap, we propose Distortion-Aware PETR (DAPETR), a projection-free detector tailored for mixed pinhole-fisheye camera setups. DAPETR incorporates two key learned-adaptive modules: a unified distortion-aware positional embedding that harmonizes positional encodings for image representations with fisheye geometry, and a bidirectional feature-geometry co-modulation module that mutually adapts image features and 3D positional embeddings. In our experiments on a converted KITTI-360 benchmark, we systematically compare our learned adaptive approach against PETR in polar coordinates (PolarPETR). We find that while both methods improve over the baseline, our learned modules achieve superior performance. Crucially, we uncover a negative interaction when combining both strategies, revealing that learned adaptation and explicit geometric reparameterization can conflict. Our final DAPETR model significantly advances the research and benchmark for fisheye BEV detection, providing critical insights into effective distortion-aware 3D perception design other than image rectification.

Related papers

• OmniPoint: Universal Monocular Metric Pointcloud from Any Camera [79.24417597712802]
  We present OmniPoint, a unified framework designed to generalize metric reconstruction across diverse imaging sensors.<n>To overcome projection rigidity, OmniPoint abandons conventional planar depth regression.<n>It instead adopts a decoupled ray and distance representation alongside a decoupled training objective.
  arXiv  Detail & Related papers  (2026-09-08T19:45:58Z)
• Geometry-Aware Fisheye-LiDAR Fusion for Robust 3D Object Detection in Low-Overlap Setups [8.422269543220251]
  We propose a Geometry-Aware Hybrid Fusion (GA-HF) framework that accounts for fisheye geometry and BEV feature distortion.<n>GA-HF is evaluated on three benchmarks: KITTI-360, Dur360BEV, and Fisheye3DOD datasets.
  arXiv  Detail & Related papers  (2026-06-07T21:26:00Z)
• QVGGT: Post-Training Quantized Visual Geometry Grounded Transformer [5.7977296539841845]
  We introduce QVGGT, a tailored quantization framework designed to compress Visual Geometry Grounded Transformer (VGGT)<n>QVGGT achieves near-lossless W4A16 quantization, preserving the accuracy of all 3D prediction heads while delivering 3$sim$4.9$times$ memory reduction and up to 2.8$times$ real hardware speedup over FP32.<n>Our approach makes high-fidelity 3D perception feasible on edge devices, enabling practical deployment of feed-forward 3D reconstruction models in real-world constrained environments.
  arXiv  Detail & Related papers  (2026-05-29T10:32:52Z)
• Benchmarking Multi-View BEV Object Detection with Mixed Pinhole and Fisheye Cameras [5.227238848375275]
  We introduce a multi-view BEV detection benchmark with mixed cameras by converting KITTI-360 into nuScenes format.<n>Our study encompasses three adaptations: rectification for zero-shot evaluation and fine-tuning of nuScenes-trained models.<n>We demonstrate that projection-free architectures are inherently more robust and effective against fisheye distortion than other VTMs.
  arXiv  Detail & Related papers  (2026-03-29T19:10:34Z)
• PFDepth: Heterogeneous Pinhole-Fisheye Joint Depth Estimation via Distortion-aware Gaussian-Splatted Volumetric Fusion [61.6340987158734]
  We present the first pinhole-fisheye framework for heterogeneous multi-view depth estimation, PFDepth.<n> PFDepth employs a unified architecture capable of processing arbitrary combinations of pinhole and fisheye cameras with varied intrinsics and extrinsics.<n>We show that PFDepth sets a state-of-the-art performance on KITTI-360 and RealHet datasets over current mainstream depth networks.
  arXiv  Detail & Related papers  (2025-09-30T09:38:59Z)
• RQR3D: Reparametrizing the regression targets for BEV-based 3D object detection [0.4604003661048266]
  Bird's-eye view (BEV)-based perception approaches have emerged as superior alternatives to perspective-based solutions.<n>We propose Restricted Quadrilateral Representation to define 3D regression targets.<n>RQR3D regresses the smallest horizontal bounding box encapsulating the oriented box, along with the offsets between the corners of these two boxes.
  arXiv  Detail & Related papers  (2025-05-23T10:52:34Z)
• Towards Generalizable Multi-Camera 3D Object Detection via Perspective Debiasing [28.874014617259935]
  Multi-Camera 3D Object Detection (MC3D-Det) has gained prominence with the advent of bird's-eye view (BEV) approaches. We propose a novel method that aligns 3D detection with 2D camera plane results, ensuring consistent and accurate detections.
  arXiv  Detail & Related papers  (2023-10-17T15:31:28Z)
• Geometric-aware Pretraining for Vision-centric 3D Object Detection [77.7979088689944]
  We propose a novel geometric-aware pretraining framework called GAPretrain. GAPretrain serves as a plug-and-play solution that can be flexibly applied to multiple state-of-the-art detectors. We achieve 46.2 mAP and 55.5 NDS on the nuScenes val set using the BEVFormer method, with a gain of 2.7 and 2.1 points, respectively.
  arXiv  Detail & Related papers  (2023-04-06T14:33:05Z)
• PolarFormer: Multi-camera 3D Object Detection with Polar Transformers [93.49713023975727]
  3D object detection in autonomous driving aims to reason "what" and "where" the objects of interest present in a 3D world. Existing methods often adopt the canonical Cartesian coordinate system with perpendicular axis. We propose a new Polar Transformer (PolarFormer) for more accurate 3D object detection in the bird's-eye-view (BEV) taking as input only multi-camera 2D images.
  arXiv  Detail & Related papers  (2022-06-30T16:32:48Z)
• Towards Model Generalization for Monocular 3D Object Detection [57.25828870799331]
  We present an effective unified camera-generalized paradigm (CGP) for Mono3D object detection. We also propose the 2D-3D geometry-consistent object scaling strategy (GCOS) to bridge the gap via an instance-level augment. Our method called DGMono3D achieves remarkable performance on all evaluated datasets and surpasses the SoTA unsupervised domain adaptation scheme.
  arXiv  Detail & Related papers  (2022-05-23T23:05:07Z)
• The KFIoU Loss for Rotated Object Detection [115.334070064346]
  In this paper, we argue that one effective alternative is to devise an approximate loss who can achieve trend-level alignment with SkewIoU loss. Specifically, we model the objects as Gaussian distribution and adopt Kalman filter to inherently mimic the mechanism of SkewIoU. The resulting new loss called KFIoU is easier to implement and works better compared with exact SkewIoU.
  arXiv  Detail & Related papers  (2022-01-29T10:54:57Z)
• Geometry-aware data augmentation for monocular 3D object detection [18.67567745336633]
  This paper focuses on monocular 3D object detection, one of the essential modules in autonomous driving systems. A key challenge is that the depth recovery problem is ill-posed in monocular data. We conduct a thorough analysis to reveal how existing methods fail to robustly estimate depth when different geometry shifts occur. We convert the aforementioned manipulations into four corresponding 3D-aware data augmentation techniques.
  arXiv  Detail & Related papers  (2021-04-12T23:12:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.