Timely Fusion of Surround Radar/Lidar for Object Detection in Autonomous Driving Systems

• URL: http://arxiv.org/abs/2309.04806v3
• Date: Mon, 27 May 2024 06:09:43 GMT
• Title: Timely Fusion of Surround Radar/Lidar for Object Detection in Autonomous Driving Systems
• Authors: Wenjing Xie, Tao Hu, Neiwen Ling, Guoliang Xing, Chun Jason Xue, Nan Guan,
• Abstract summary: Fusing Radar and Lidar sensor data can fully utilize their complementary advantages and provide more accurate reconstruction of the surrounding. Existing Radar/Lidar fusion methods have to work at the low frequency of surround Radar. This paper develops techniques to fuse surround Radar/Lidar with working frequency only limited by the faster surround Lidar.
• Score: 13.998883144668941
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Fusing Radar and Lidar sensor data can fully utilize their complementary advantages and provide more accurate reconstruction of the surrounding for autonomous driving systems. Surround Radar/Lidar can provide 360-degree view sampling with the minimal cost, which are promising sensing hardware solutions for autonomous driving systems. However, due to the intrinsic physical constraints, the rotating speed of surround Radar, and thus the frequency to generate Radar data frames, is much lower than surround Lidar. Existing Radar/Lidar fusion methods have to work at the low frequency of surround Radar, which cannot meet the high responsiveness requirement of autonomous driving systems.This paper develops techniques to fuse surround Radar/Lidar with working frequency only limited by the faster surround Lidar instead of the slower surround Radar, based on the state-of-the-art object detection model MVDNet. The basic idea of our approach is simple: we let MVDNet work with temporally unaligned data from Radar/Lidar, so that fusion can take place at any time when a new Lidar data frame arrives, instead of waiting for the slow Radar data frame. However, directly applying MVDNet to temporally unaligned Radar/Lidar data greatly degrades its object detection accuracy. The key information revealed in this paper is that we can achieve high output frequency with little accuracy loss by enhancing the training procedure to explore the temporal redundancy in MVDNet so that it can tolerate the temporal unalignment of input data. We explore several different ways of training enhancement and compare them quantitatively with experiments.

Related papers

• Radar Fields: Frequency-Space Neural Scene Representations for FMCW Radar [62.51065633674272]
  We introduce Radar Fields - a neural scene reconstruction method designed for active radar imagers. Our approach unites an explicit, physics-informed sensor model with an implicit neural geometry and reflectance model to directly synthesize raw radar measurements. We validate the effectiveness of the method across diverse outdoor scenarios, including urban scenes with dense vehicles and infrastructure.
  arXiv  Detail & Related papers  (2024-05-07T20:44:48Z)
• Bootstrapping Autonomous Driving Radars with Self-Supervised Learning [13.13679517730015]
  Training radar models is hindered by the cost and difficulty of annotating large-scale radar data. We propose a self-supervised learning framework to leverage the large amount of unlabeled radar data to pre-train radar-only embeddings for self-driving perception tasks. When used for downstream object detection, we demonstrate that the proposed self-supervision framework can improve the accuracy of state-of-the-art supervised baselines by $5.8%$ in mAP.
  arXiv  Detail & Related papers  (2023-12-07T18:38:39Z)
• Echoes Beyond Points: Unleashing the Power of Raw Radar Data in Multi-modality Fusion [74.84019379368807]
  We propose a novel method named EchoFusion to skip the existing radar signal processing pipeline. Specifically, we first generate the Bird's Eye View (BEV) queries and then take corresponding spectrum features from radar to fuse with other sensors.
  arXiv  Detail & Related papers  (2023-07-31T09:53:50Z)
• Bi-LRFusion: Bi-Directional LiDAR-Radar Fusion for 3D Dynamic Object Detection [78.59426158981108]
  We introduce a bi-directional LiDAR-Radar fusion framework, termed Bi-LRFusion, to tackle the challenges and improve 3D detection for dynamic objects. We conduct extensive experiments on nuScenes and ORR datasets, and show that our Bi-LRFusion achieves state-of-the-art performance for detecting dynamic objects.
  arXiv  Detail & Related papers  (2023-06-02T10:57:41Z)
• RadarFormer: Lightweight and Accurate Real-Time Radar Object Detection Model [13.214257841152033]
  Radar-centric data sets do not get a lot of attention in the development of deep learning techniques for radar perception. We propose a transformers-based model, named RadarFormer, that utilizes state-of-the-art developments in vision deep learning. Our model also introduces a channel-chirp-time merging module that reduces the size and complexity of our models by more than 10 times without compromising accuracy.
  arXiv  Detail & Related papers  (2023-04-17T17:07:35Z)
• Complex-valued Convolutional Neural Networks for Enhanced Radar Signal Denoising and Interference Mitigation [73.0103413636673]
  We propose the use of Complex-Valued Convolutional Neural Networks (CVCNNs) to address the issue of mutual interference between radar sensors. CVCNNs increase data efficiency, speeds up network training and substantially improves the conservation of phase information during interference removal.
  arXiv  Detail & Related papers  (2021-04-29T10:06:29Z)
• LiRaNet: End-to-End Trajectory Prediction using Spatio-Temporal Radar Fusion [52.59664614744447]
  We present LiRaNet, a novel end-to-end trajectory prediction method which utilizes radar sensor information along with widely used lidar and high definition (HD) maps. automotive radar provides rich, complementary information, allowing for longer range vehicle detection as well as instantaneous velocity measurements.
  arXiv  Detail & Related papers  (2020-10-02T00:13:00Z)
• RadarNet: Exploiting Radar for Robust Perception of Dynamic Objects [73.80316195652493]
  We tackle the problem of exploiting Radar for perception in the context of self-driving cars. We propose a new solution that exploits both LiDAR and Radar sensors for perception. Our approach, dubbed RadarNet, features a voxel-based early fusion and an attention-based late fusion.
  arXiv  Detail & Related papers  (2020-07-28T17:15:02Z)
• Probabilistic Oriented Object Detection in Automotive Radar [8.281391209717103]
  We propose a deep-learning based algorithm for radar object detection. We created a new multimodal dataset with 102544 frames of raw radar and synchronized LiDAR data. Our best performing radar detection model achieves 77.28% AP under oriented IoU of 0.3.
  arXiv  Detail & Related papers  (2020-04-11T05:29:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.