FisheyeDetNet: 360° Surround view Fisheye Camera based Object Detection System for Autonomous Driving

• URL: http://arxiv.org/abs/2404.13443v2
• Date: Sat, 27 Apr 2024 14:02:35 GMT
• Title: FisheyeDetNet: 360° Surround view Fisheye Camera based Object Detection System for Autonomous Driving
• Authors: Ganesh Sistu, Senthil Yogamani,
• Abstract summary: Object detection is a mature problem in autonomous driving with pedestrian detection being one of the first deployed algorithms. Standard bounding box representation fails in fisheye cameras due to heavy radial distortion, particularly in the periphery. We design rotated bounding boxes, ellipse, generic polygon as polar arc/angle representations and define an instance segmentation mIOU metric to analyze these representations. The proposed model FisheyeDetNet with polygon outperforms others and achieves a mAP score of 49.5 % on Valeo fisheye surround-view dataset for automated driving applications.
• Score: 4.972459365804512
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Object detection is a mature problem in autonomous driving with pedestrian detection being one of the first deployed algorithms. It has been comprehensively studied in the literature. However, object detection is relatively less explored for fisheye cameras used for surround-view near field sensing. The standard bounding box representation fails in fisheye cameras due to heavy radial distortion, particularly in the periphery. To mitigate this, we explore extending the standard object detection output representation of bounding box. We design rotated bounding boxes, ellipse, generic polygon as polar arc/angle representations and define an instance segmentation mIOU metric to analyze these representations. The proposed model FisheyeDetNet with polygon outperforms others and achieves a mAP score of 49.5 % on Valeo fisheye surround-view dataset for automated driving applications. This dataset has 60K images captured from 4 surround-view cameras across Europe, North America and Asia. To the best of our knowledge, this is the first detailed study on object detection on fisheye cameras for autonomous driving scenarios.

Related papers

• Streaming Object Detection on Fisheye Cameras for Automatic Parking [0.0]
  We propose a real-time detection framework equipped with a dualflow perception module that can predict the future and alleviate the time-lag problem. The standard bounding box is unsuitable for the object in fisheye camera images due to the strong radial distortion of the fisheye camera. We propose a new periodic angle loss function to regress the angle of the box, which is the simple and accurate representation method of objects.
  arXiv  Detail & Related papers  (2023-05-24T04:30:25Z)
• Surround-view Fisheye Camera Perception for Automated Driving: Overview, Survey and Challenges [1.4452405977630436]
  Four fisheye cameras on four sides of the vehicle are sufficient to cover 360deg around the vehicle capturing the entire near-field region. Some primary use cases are automated parking, traffic jam assist, and urban driving. Due to the large radial distortion of fisheye cameras, standard algorithms can not be extended easily to the surround-view use case.
  arXiv  Detail & Related papers  (2022-05-26T11:38:04Z)
• SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimation [101.55622133406446]
  We propose a SurroundDepth method to incorporate the information from multiple surrounding views to predict depth maps across cameras. Specifically, we employ a joint network to process all the surrounding views and propose a cross-view transformer to effectively fuse the information from multiple views. In experiments, our method achieves the state-of-the-art performance on the challenging multi-camera depth estimation datasets.
  arXiv  Detail & Related papers  (2022-04-07T17:58:47Z)
• Rope3D: TheRoadside Perception Dataset for Autonomous Driving and Monocular 3D Object Detection Task [48.555440807415664]
  We present the first high-diversity challenging Roadside Perception 3D dataset- Rope3D from a novel view. The dataset consists of 50k images and over 1.5M 3D objects in various scenes. We propose to leverage the geometry constraint to solve the inherent ambiguities caused by various sensors, viewpoints.
  arXiv  Detail & Related papers  (2022-03-25T12:13:23Z)
• SVDistNet: Self-Supervised Near-Field Distance Estimation on Surround View Fisheye Cameras [30.480562747903186]
  A 360deg perception of scene geometry is essential for automated driving, notably for parking and urban driving scenarios. We present novel camera-geometry adaptive multi-scale convolutions which utilize the camera parameters as a conditional input. We evaluate our approach on the Fisheye WoodScape surround-view dataset, significantly improving over previous approaches.
  arXiv  Detail & Related papers  (2021-04-09T15:20:20Z)
• OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving [10.3540046389057]
  This work presents a multi-task visual perception network on unrectified fisheye images. It consists of six primary tasks necessary for an autonomous driving system. We demonstrate that the jointly trained model performs better than the respective single task versions.
  arXiv  Detail & Related papers  (2021-02-15T10:46:24Z)
• Generalized Object Detection on Fisheye Cameras for Autonomous Driving: Dataset, Representations and Baseline [5.1450366450434295]
  We explore better representations like oriented bounding box, ellipse, and generic polygon for object detection in fisheye images. We design a novel curved bounding box model that has optimal properties for fisheye distortion models. It is the first detailed study on object detection on fisheye cameras for autonomous driving scenarios.
  arXiv  Detail & Related papers  (2020-12-03T18:00:16Z)
• Towards Autonomous Driving: a Multi-Modal 360$^{\circ}$ Perception Proposal [87.11988786121447]
  This paper presents a framework for 3D object detection and tracking for autonomous vehicles. The solution, based on a novel sensor fusion configuration, provides accurate and reliable road environment detection. A variety of tests of the system, deployed in an autonomous vehicle, have successfully assessed the suitability of the proposed perception stack.
  arXiv  Detail & Related papers  (2020-08-21T20:36:21Z)
• Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D [100.93808824091258]
  We propose a new end-to-end architecture that directly extracts a bird's-eye-view representation of a scene given image data from an arbitrary number of cameras. Our approach is to "lift" each image individually into a frustum of features for each camera, then "splat" all frustums into a bird's-eye-view grid. We show that the representations inferred by our model enable interpretable end-to-end motion planning by "shooting" template trajectories into a bird's-eye-view cost map output by our network.
  arXiv  Detail & Related papers  (2020-08-13T06:29:01Z)
• DOPS: Learning to Detect 3D Objects and Predict their 3D Shapes [54.239416488865565]
  We propose a fast single-stage 3D object detection method for LIDAR data. The core novelty of our method is a fast, single-pass architecture that both detects objects in 3D and estimates their shapes. We find that our proposed method achieves state-of-the-art results by 5% on object detection in ScanNet scenes, and it gets top results by 3.4% in the Open dataset.
  arXiv  Detail & Related papers  (2020-04-02T17:48:50Z)
• BirdNet+: End-to-End 3D Object Detection in LiDAR Bird's Eye View [117.44028458220427]
  On-board 3D object detection in autonomous vehicles often relies on geometry information captured by LiDAR devices. We present a fully end-to-end 3D object detection framework that can infer oriented 3D boxes solely from BEV images.
  arXiv  Detail & Related papers  (2020-03-09T15:08:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.