LiDAR Cluster First and Camera Inference Later: A New Perspective Towards Autonomous Driving

• URL: http://arxiv.org/abs/2111.09799v2
• Date: Fri, 19 Nov 2021 15:24:51 GMT
• Title: LiDAR Cluster First and Camera Inference Later: A New Perspective Towards Autonomous Driving
• Authors: Jiyang Chen, Simon Yu, Rohan Tabish, Ayoosh Bansal, Shengzhong Liu, Tarek Abdelzaher, and Lui Sha
• Abstract summary: We present a new end-to-end pipeline for Autonomous Vehicles (AV) that introduces the concept of LiDAR cluster first and camera inference later to detect and classify objects. First, our pipeline prioritizes detecting objects that pose a higher risk of collision to the AV, giving more time for the AV to react to unsafe conditions. We show that our novel object detection pipeline prioritizes the detection of higher risk objects while simultaneously achieving comparable accuracy and a 25% higher average speed compared to camera inference only.
• Score: 3.7678372667663393
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Object detection in state-of-the-art Autonomous Vehicles (AV) framework relies heavily on deep neural networks. Typically, these networks perform object detection uniformly on the entire camera LiDAR frames. However, this uniformity jeopardizes the safety of the AV by giving the same priority to all objects in the scenes regardless of their risk of collision to the AV. In this paper, we present a new end-to-end pipeline for AV that introduces the concept of LiDAR cluster first and camera inference later to detect and classify objects. The benefits of our proposed framework are twofold. First, our pipeline prioritizes detecting objects that pose a higher risk of collision to the AV, giving more time for the AV to react to unsafe conditions. Second, it also provides, on average, faster inference speeds compared to popular deep neural network pipelines. We design our framework using the real-world datasets, the Waymo Open Dataset, solving challenges arising from the limitations of LiDAR sensors and object detection algorithms. We show that our novel object detection pipeline prioritizes the detection of higher risk objects while simultaneously achieving comparable accuracy and a 25% higher average speed compared to camera inference only.

Related papers

• A Real-Time Defense Against Object Vanishing Adversarial Patch Attacks for Object Detection in Autonomous Vehicles [0.0]
  ADAV (Adversarial Defense for Autonomous Vehicles) is a novel defense methodology against object vanishing patch attacks. ADAV runs in real-time and leverages contextual information from prior frames in an AV's video feed. ADAV is evaluated using real-world driving data from the Berkeley Deep Drive BDD100K dataset.
  arXiv  Detail & Related papers  (2024-12-09T05:21:14Z)
• FuzzRisk: Online Collision Risk Estimation for Autonomous Vehicles based on Depth-Aware Object Detection via Fuzzy Inference [6.856508678236828]
  The framework takes two sets of predictions from different algorithms and associates their inconsistencies with the collision risk via fuzzy inference. We experimentally validate that, based on Intersection-over-Union (IoU) and a depth discrepancy measure, the inconsistencies between the two sets of predictions strongly correlate to the error of the 3D object detector. We optimize the fuzzy inference system towards an existing offline metric that matches AV collision rates well.
  arXiv  Detail & Related papers  (2024-11-09T20:20:36Z)
• Towards Unsupervised Object Detection From LiDAR Point Clouds [46.57452180314863]
  OYSTER (Object Discovery via Spatio-Temporal Refinement) is able to detect objects in a zero-shot manner without supervised finetuning. We propose a new planning-centric perception metric based on distance-to-collision.
  arXiv  Detail & Related papers  (2023-11-03T16:12:01Z)
• Multi-Task Cross-Modality Attention-Fusion for 2D Object Detection [6.388430091498446]
  We propose two new radar preprocessing techniques to better align radar and camera data. We also introduce a Multi-Task Cross-Modality Attention-Fusion Network (MCAF-Net) for object detection. Our approach outperforms current state-of-the-art radar-camera fusion-based object detectors in the nuScenes dataset.
  arXiv  Detail & Related papers  (2023-07-17T09:26:13Z)
• Long Range Object-Level Monocular Depth Estimation for UAVs [0.0]
  We propose several novel extensions to state-of-the-art methods for monocular object detection from images at long range. Firstly, we propose Sigmoid and ReLU-like encodings when modeling depth estimation as a regression task. Secondly, we frame the depth estimation as a classification problem and introduce a Soft-Argmax function in the calculation of the training loss.
  arXiv  Detail & Related papers  (2023-02-17T15:26:04Z)
• Threatening Patch Attacks on Object Detection in Optical Remote Sensing Images [55.09446477517365]
  Advanced Patch Attacks (PAs) on object detection in natural images have pointed out the great safety vulnerability in methods based on deep neural networks. We propose a more Threatening PA without the scarification of the visual quality, dubbed TPA. To the best of our knowledge, this is the first attempt to study the PAs on object detection in O-RSIs, and we hope this work can get our readers interested in studying this topic.
  arXiv  Detail & Related papers  (2023-02-13T02:35:49Z)
• NVRadarNet: Real-Time Radar Obstacle and Free Space Detection for Autonomous Driving [57.03126447713602]
  We present a deep neural network (DNN) that detects dynamic obstacles and drivable free space using automotive RADAR sensors. The network runs faster than real time on an embedded GPU and shows good generalization across geographic regions.
  arXiv  Detail & Related papers  (2022-09-29T01:30:34Z)
• Fully Convolutional One-Stage 3D Object Detection on LiDAR Range Images [96.66271207089096]
  FCOS-LiDAR is a fully convolutional one-stage 3D object detector for LiDAR point clouds of autonomous driving scenes. We show that an RV-based 3D detector with standard 2D convolutions alone can achieve comparable performance to state-of-the-art BEV-based detectors.
  arXiv  Detail & Related papers  (2022-05-27T05:42:16Z)
• FOVEA: Foveated Image Magnification for Autonomous Navigation [53.69803081925454]
  We propose an attentional approach that elastically magnifies certain regions while maintaining a small input canvas. Our proposed method boosts the detection AP over standard Faster R-CNN, with and without finetuning. On the autonomous driving datasets Argoverse-HD and BDD100K, we show our proposed method boosts the detection AP over standard Faster R-CNN, with and without finetuning.
  arXiv  Detail & Related papers  (2021-08-27T03:07:55Z)
• Fast Motion Understanding with Spatiotemporal Neural Networks and Dynamic Vision Sensors [99.94079901071163]
  This paper presents a Dynamic Vision Sensor (DVS) based system for reasoning about high speed motion. We consider the case of a robot at rest reacting to a small, fast approaching object at speeds higher than 15m/s. We highlight the results of our system to a toy dart moving at 23.4m/s with a 24.73deg error in $theta$, 18.4mm average discretized radius prediction error, and 25.03% median time to collision prediction error.
  arXiv  Detail & Related papers  (2020-11-18T17:55:07Z)
• RadarNet: Exploiting Radar for Robust Perception of Dynamic Objects [73.80316195652493]
  We tackle the problem of exploiting Radar for perception in the context of self-driving cars. We propose a new solution that exploits both LiDAR and Radar sensors for perception. Our approach, dubbed RadarNet, features a voxel-based early fusion and an attention-based late fusion.
  arXiv  Detail & Related papers  (2020-07-28T17:15:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.