Exploring Surround-View Fisheye Camera 3D Object Detection

• URL: http://arxiv.org/abs/2511.18695v1
• Date: Mon, 24 Nov 2025 02:28:56 GMT
• Title: Exploring Surround-View Fisheye Camera 3D Object Detection
• Abstract summary: We investigate the technical feasibility of implementing end-to-end 3D object detection with surround-view fisheye camera system.<n>We develop two methods that incorporate the unique geometry of fisheye images into mainstream detection frameworks.<n> Experiments on Fisheye3DOD show that our fisheye-compatible modeling improves accuracy by up to 6.2% over baseline methods.
• Score: 44.09164098759971
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this work, we explore the technical feasibility of implementing end-to-end 3D object detection (3DOD) with surround-view fisheye camera system. Specifically, we first investigate the performance drop incurred when transferring classic pinhole-based 3D object detectors to fisheye imagery. To mitigate this, we then develop two methods that incorporate the unique geometry of fisheye images into mainstream detection frameworks: one based on the bird's-eye-view (BEV) paradigm, named FisheyeBEVDet, and the other on the query-based paradigm, named FisheyePETR. Both methods adopt spherical spatial representations to effectively capture fisheye geometry. In light of the lack of dedicated evaluation benchmarks, we release Fisheye3DOD, a new open dataset synthesized using CARLA and featuring both standard pinhole and fisheye camera arrays. Experiments on Fisheye3DOD show that our fisheye-compatible modeling improves accuracy by up to 6.2% over baseline methods.

Related papers

• OmniPoint: Universal Monocular Metric Pointcloud from Any Camera [79.24417597712802]
  We present OmniPoint, a unified framework designed to generalize metric reconstruction across diverse imaging sensors.<n>To overcome projection rigidity, OmniPoint abandons conventional planar depth regression.<n>It instead adopts a decoupled ray and distance representation alongside a decoupled training objective.
  arXiv  Detail & Related papers  (2026-09-08T19:45:58Z)
• MIVIFI: Bridging Perspective and Fisheye Domains for Training Multi-View Fisheye Image Generation Models [44.93230202195852]
  Fisheye cameras offer a cost-effective solution by enabling full surround coverage with as few as two sensors.<n>Existing multi-view fisheye datasets are limited, and rare corner cases typically require computationally expensive 3D simulations.<n>We present two distinct methods for generating multi-view fisheye images conditioned on volumetric semantic representations.
  arXiv  Detail & Related papers  (2026-08-24T11:46:28Z)
• WideDepth: Millimeter-Accurate Benchmark for Fisheye Depth Estimation [3.449965668627019]
  We introduce WideDepth - the first indoor dataset for fisheye depth estimation.<n>Our dataset includes 101 scenes containing 5K high-resolution stereo pairs labeled with millimeter-level ground truth depth and disparity.<n>We propose a method to adapt pinhole-trained stereo models to fisheye images and introduce a novel stereo fisheye image generation pipeline.
  arXiv  Detail & Related papers  (2026-05-22T15:39:10Z)
• FishRoPE: Projective Rotary Position Embeddings for Omnidirectional Visual Perception [6.114308991453516]
  Vision foundation models (VFMs) and Bird's Eye View (BEV) representation assume the rectilinear geometry of pinhole cameras.<n>ours is a lightweight framework that adapts frozen VFMs to fisheye geometry through two components.
  arXiv  Detail & Related papers  (2026-04-12T00:46:51Z)
• Fisheye3R: Adapting Unified 3D Feed-Forward Foundation Models to Fisheye Lenses [5.987299558427844]
  Fisheye3R is a novel adaptation framework that extends multi-view 3D reconstruction foundation models to accommodate fisheye inputs without performance regression on perspective images.<n>Our approach consistently improves camera pose, depth, point map, and field-of-view estimation on fisheye images.
  arXiv  Detail & Related papers  (2026-03-30T18:18:56Z)
• Fisheye-GS: Lightweight and Extensible Gaussian Splatting Module for Fisheye Cameras [17.75361921797657]
  We introduce Fisheye-GS.This innovative method recalculates the projection transformation and its gradients for fisheye cameras. Our approach can be seamlessly integrated as a module into other efficient 3D rendering methods.
  arXiv  Detail & Related papers  (2024-09-07T07:53:40Z)
• VFMM3D: Releasing the Potential of Image by Vision Foundation Model for Monocular 3D Object Detection [80.62052650370416]
  monocular 3D object detection holds significant importance across various applications, including autonomous driving and robotics. In this paper, we present VFMM3D, an innovative framework that leverages the capabilities of Vision Foundation Models (VFMs) to accurately transform single-view images into LiDAR point cloud representations.
  arXiv  Detail & Related papers  (2024-04-15T03:12:12Z)
• SDGE: Stereo Guided Depth Estimation for 360$^\circ$ Camera Sets [65.64958606221069]
  Multi-camera systems are often used in autonomous driving to achieve a 360$circ$ perception. These 360$circ$ camera sets often have limited or low-quality overlap regions, making multi-view stereo methods infeasible for the entire image. We propose the Stereo Guided Depth Estimation (SGDE) method, which enhances depth estimation of the full image by explicitly utilizing multi-view stereo results on the overlap.
  arXiv  Detail & Related papers  (2024-02-19T02:41:37Z)
• CAPE: Camera View Position Embedding for Multi-View 3D Object Detection [100.02565745233247]
  Current query-based methods rely on global 3D position embeddings to learn the geometric correspondence between images and 3D space. We propose a novel method based on CAmera view Position Embedding, called CAPE. CAPE achieves state-of-the-art performance (61.0% NDS and 52.5% mAP) among all LiDAR-free methods on nuScenes dataset.
  arXiv  Detail & Related papers  (2023-03-17T18:59:54Z)
• FisheyeEX: Polar Outpainting for Extending the FoV of Fisheye Lens [84.12722334460022]
  Fisheye lens gains increasing applications in computational photography and assisted driving because of its wide field of view (FoV) In this paper, we present a FisheyeEX method that extends the FoV of the fisheye lens by outpainting the invalid regions. The results demonstrate that our approach significantly outperforms the state-of-the-art methods, gaining around 27% more content beyond the original fisheye image.
  arXiv  Detail & Related papers  (2022-06-12T21:38:50Z)
• Learning Geometry-Guided Depth via Projective Modeling for Monocular 3D Object Detection [70.71934539556916]
  We learn geometry-guided depth estimation with projective modeling to advance monocular 3D object detection. Specifically, a principled geometry formula with projective modeling of 2D and 3D depth predictions in the monocular 3D object detection network is devised. Our method remarkably improves the detection performance of the state-of-the-art monocular-based method without extra data by 2.80% on the moderate test setting.
  arXiv  Detail & Related papers  (2021-07-29T12:30:39Z)
• Disentangling and Vectorization: A 3D Visual Perception Approach for Autonomous Driving Based on Surround-View Fisheye Cameras [3.485767750936058]
  Multidimensional Vector is proposed to include the utilizable information generated in different dimensions and stages. The experiments of real fisheye images demonstrate that our solution achieves state-of-the-art accuracy while being real-time in practice.
  arXiv  Detail & Related papers  (2021-07-19T13:24:21Z)
• FisheyeSuperPoint: Keypoint Detection and Description Network for Fisheye Images [2.187613144178315]
  Keypoint detection and description is a commonly used building block in computer vision systems. SuperPoint is a self-supervised keypoint detector and descriptor that has achieved state-of-the-art results on homography estimation. We introduce a fisheye adaptation pipeline to enable training on undistorted fisheye images.
  arXiv  Detail & Related papers  (2021-02-27T11:26:34Z)
• 3D Object Detection from a Single Fisheye Image Without a Single Fisheye Training Image [7.86363825307044]
  We show how to use existing monocular 3D object detection models, trained only on rectilinear images, to detect 3D objects in images from fisheye cameras. We outperform the only existing method for monocular 3D object detection in panoramas on a benchmark of synthetic data.
  arXiv  Detail & Related papers  (2020-03-08T11:03:05Z)
• ZoomNet: Part-Aware Adaptive Zooming Neural Network for 3D Object Detection [69.68263074432224]
  We present a novel framework named ZoomNet for stereo imagery-based 3D detection. The pipeline of ZoomNet begins with an ordinary 2D object detection model which is used to obtain pairs of left-right bounding boxes. To further exploit the abundant texture cues in RGB images for more accurate disparity estimation, we introduce a conceptually straight-forward module -- adaptive zooming.
  arXiv  Detail & Related papers  (2020-03-01T17:18:08Z)
• SilhoNet-Fisheye: Adaptation of A ROI Based Object Pose Estimation Network to Monocular Fisheye Images [15.573003283204958]
  We present a novel framework for adapting a ROI-based 6D object pose estimation method to work on full fisheye images. We also contribute a fisheye image dataset, called UWHandles, with 6D object pose and 2D bounding box annotations.
  arXiv  Detail & Related papers  (2020-02-27T19:57:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.