Geometry-Aware Fisheye-LiDAR Fusion for Robust 3D Object Detection in Low-Overlap Setups

• URL: http://arxiv.org/abs/2606.08844v1
• Date: Sun, 07 Jun 2026 21:26:00 GMT
• Title: Geometry-Aware Fisheye-LiDAR Fusion for Robust 3D Object Detection in Low-Overlap Setups
• Abstract summary: We propose a Geometry-Aware Hybrid Fusion (GA-HF) framework that accounts for fisheye geometry and BEV feature distortion.<n>GA-HF is evaluated on three benchmarks: KITTI-360, Dur360BEV, and Fisheye3DOD datasets.
• Score: 8.422269543220251
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As autonomous systems expand from capital-intensive robotaxis to cost-sensitive logistics, sensor configurations are increasingly optimized for coverage-per-cost. A prevalent sparse-view setup utilizes dual-fisheye cameras with a roof-mounted LiDAR, introducing severe geometric challenges: extreme radial distortion, minimal overlap, and misalignment between spherical projections and rectilinear grids. BEV fusion algorithms typically force image and point cloud modalities into unified Cartesian grids early in the pipeline, causing significant feature distortion and information loss for wide-view fisheye cameras. To address this, we propose a Geometry-Aware Hybrid Fusion (GA-HF) framework that explicitly accounts for fisheye geometry and BEV feature distortion, where fisheye features are lifted into a polar BEV grid via a Distortion-Aware Lift-Splat-Shoot (LSS) module to preserve native angular density, while LiDAR features are processed in native Cartesian space for metric fidelity of bounding box regression. To bridge these heterogeneous streams, we introduce a Dual-Attention Warping Correction module that applies spatial and channel attention to the warped camera features before fusion, explicitly suppressing artifacts in low-quality peripheral regions while enhancing high-quality semantic cues. GA-HF is evaluated on three benchmarks: KITTI-360, Dur360BEV, and Fisheye3DOD datasets. To the best of our knowledge, it is the first approach to explore LiDAR-fisheye camera fusion. On KITTI-360, GA-HF improves NDS by 4.2% over Cartesian baselines; on Dur360BEV, it surpasses both LiDAR-only and BEVFusion, while significantly reducing orientation error despite the geometric distortions; on Fisheye3DOD, it attains the highest detection score among all fusion methods.

Related papers

• OmniPoint: Universal Monocular Metric Pointcloud from Any Camera [79.24417597712802]
  We present OmniPoint, a unified framework designed to generalize metric reconstruction across diverse imaging sensors.<n>To overcome projection rigidity, OmniPoint abandons conventional planar depth regression.<n>It instead adopts a decoupled ray and distance representation alongside a decoupled training objective.
  arXiv  Detail & Related papers  (2026-09-08T19:45:58Z)
• Distortion-Aware PETR for BEV Object Detection with Mixed Pinhole-Fisheye Cameras [0.0]
  Fisheye cameras are widely deployed in autonomous driving perception suites for their low cost and full-coverage field of view (FOV)<n>Severe radial distortion challenges most BEV detectors by violating the fundamental assumption of uniform sampling.<n>We propose Distortion-Aware PETR (DAPETR), a projection-free detector tailored for mixed pinhole-fisheye camera setups.
  arXiv  Detail & Related papers  (2026-06-07T15:32:10Z)
• FishRoPE: Projective Rotary Position Embeddings for Omnidirectional Visual Perception [6.114308991453516]
  Vision foundation models (VFMs) and Bird's Eye View (BEV) representation assume the rectilinear geometry of pinhole cameras.<n>ours is a lightweight framework that adapts frozen VFMs to fisheye geometry through two components.
  arXiv  Detail & Related papers  (2026-04-12T00:46:51Z)
• DirectFisheye-GS: Enabling Native Fisheye Input in Gaussian Splatting with Cross-View Joint Optimization [17.91043282211744]
  3D Gaussian Splatting (3DGS) has enabled efficient 3D scene reconstruction from everyday images with real-time, high-fidelity rendering.<n>Fisheye cameras, with their wider field of view (FOV), promise high-quality reconstructions from fewer inputs.
  arXiv  Detail & Related papers  (2026-04-01T09:00:04Z)
• Enhancing Underwater Light Field Images via Global Geometry-aware Diffusion Process [93.00033672476206]
  GeoDiff-LF is a novel diffusion-based framework built upon SD-Turbo to enhance underwater 4-D LF imaging.<n>By integrating diffusion priors and LF geometry, GeoDiff-LF effectively mitigates color distortion in underwater scenes.
  arXiv  Detail & Related papers  (2026-01-29T02:27:22Z)
• Robust Mesh Saliency GT Acquisition in VR via View Cone Sampling and Geometric Smoothing [59.12032628787018]
  3D mesh saliency ground truth is essential for human-centric visual modeling in virtual reality (VR)<n>Current VR eye-tracking pipelines rely on single ray sampling and Euclidean smoothing, triggering texture attention and signal leakage across gaps.<n>This paper proposes a robust framework to address these limitations.
  arXiv  Detail & Related papers  (2026-01-06T05:20:12Z)
• FisheyeGaussianLift: BEV Feature Lifting for Surround-View Fisheye Camera Perception [0.8374040635931297]
  We present a distortion-aware BEV segmentation framework that processes multi-camera high-resolution fisheye images.<n>Each image pixel is lifted into 3D space via Gaussian parameterization, predicting spatial means and anisotropic covariances to explicitly model geometric uncertainty.<n>Experiments demonstrate strong segmentation performance on complex parking and urban driving scenarios, achieving IoU scores of 87.75% for drivable regions and 57.26% for vehicles under severe fisheye distortion.
  arXiv  Detail & Related papers  (2025-11-21T12:42:07Z)
• PFDepth: Heterogeneous Pinhole-Fisheye Joint Depth Estimation via Distortion-aware Gaussian-Splatted Volumetric Fusion [61.6340987158734]
  We present the first pinhole-fisheye framework for heterogeneous multi-view depth estimation, PFDepth.<n> PFDepth employs a unified architecture capable of processing arbitrary combinations of pinhole and fisheye cameras with varied intrinsics and extrinsics.<n>We show that PFDepth sets a state-of-the-art performance on KITTI-360 and RealHet datasets over current mainstream depth networks.
  arXiv  Detail & Related papers  (2025-09-30T09:38:59Z)
• GEOcc: Geometrically Enhanced 3D Occupancy Network with Implicit-Explicit Depth Fusion and Contextual Self-Supervision [49.839374549646884]
  This paper presents GEOcc, a Geometric-Enhanced Occupancy network tailored for vision-only surround-view perception.<n>Our approach achieves State-Of-The-Art performance on the Occ3D-nuScenes dataset with the least image resolution needed and the most weightless image backbone.
  arXiv  Detail & Related papers  (2024-05-17T07:31:20Z)
• MSMDFusion: Fusing LiDAR and Camera at Multiple Scales with Multi-Depth Seeds for 3D Object Detection [89.26380781863665]
  Fusing LiDAR and camera information is essential for achieving accurate and reliable 3D object detection in autonomous driving systems. Recent approaches aim at exploring the semantic densities of camera features through lifting points in 2D camera images into 3D space for fusion. We propose a novel framework that focuses on the multi-scale progressive interaction of the multi-granularity LiDAR and camera features.
  arXiv  Detail & Related papers  (2022-09-07T12:29:29Z)
• DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection [83.18142309597984]
  Lidars and cameras are critical sensors that provide complementary information for 3D detection in autonomous driving. We develop a family of generic multi-modal 3D detection models named DeepFusion, which is more accurate than previous methods.
  arXiv  Detail & Related papers  (2022-03-15T18:46:06Z)
• 3D-CVF: Generating Joint Camera and LiDAR Features Using Cross-View Spatial Feature Fusion for 3D Object Detection [10.507404260449333]
  We propose a new architecture for fusing camera and LiDAR sensors for 3D object detection. The proposed 3D-CVF achieves state-of-the-art performance in the KITTI benchmark.
  arXiv  Detail & Related papers  (2020-04-27T08:34:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.