Echoes Beyond Points: Unleashing the Power of Raw Radar Data in Multi-modality Fusion

• URL: http://arxiv.org/abs/2307.16532v2
• Date: Mon, 13 Nov 2023 09:48:03 GMT
• Title: Echoes Beyond Points: Unleashing the Power of Raw Radar Data in Multi-modality Fusion
• Authors: Yang Liu, Feng Wang, Naiyan Wang, Zhaoxiang Zhang
• Abstract summary: We propose a novel method named EchoFusion to skip the existing radar signal processing pipeline. Specifically, we first generate the Bird's Eye View (BEV) queries and then take corresponding spectrum features from radar to fuse with other sensors.
• Score: 74.84019379368807
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Radar is ubiquitous in autonomous driving systems due to its low cost and good adaptability to bad weather. Nevertheless, the radar detection performance is usually inferior because its point cloud is sparse and not accurate due to the poor azimuth and elevation resolution. Moreover, point cloud generation algorithms already drop weak signals to reduce the false targets which may be suboptimal for the use of deep fusion. In this paper, we propose a novel method named EchoFusion to skip the existing radar signal processing pipeline and then incorporate the radar raw data with other sensors. Specifically, we first generate the Bird's Eye View (BEV) queries and then take corresponding spectrum features from radar to fuse with other sensors. By this approach, our method could utilize both rich and lossless distance and speed clues from radar echoes and rich semantic clues from images, making our method surpass all existing methods on the RADIal dataset, and approach the performance of LiDAR. The code will be released on https://github.com/tusen-ai/EchoFusion.

Related papers

• A Resource Efficient Fusion Network for Object Detection in Bird's-Eye View using Camera and Raw Radar Data [7.2508100569856975]
  We use the raw range-Doppler spectrum of radar data to process camera images. We extract the corresponding features with our camera encoder-decoder architecture. The resultant feature maps are fused with Range-Azimuth features, recovered from the RD spectrum input to perform object detection.
  arXiv  Detail & Related papers  (2024-11-20T13:26:13Z)
• SparseRadNet: Sparse Perception Neural Network on Subsampled Radar Data [5.344444942640663]
  Radar raw data often contains excessive noise, whereas radar point clouds retain only limited information. We introduce an adaptive subsampling method together with a tailored network architecture that exploits the sparsity patterns. Experiments on the RADIal dataset show that our SparseRadNet exceeds state-of-the-art (SOTA) performance in object detection and achieves close to SOTA accuracy in freespace segmentation.
  arXiv  Detail & Related papers  (2024-06-15T11:26:10Z)
• Radar Fields: Frequency-Space Neural Scene Representations for FMCW Radar [62.51065633674272]
  We introduce Radar Fields - a neural scene reconstruction method designed for active radar imagers. Our approach unites an explicit, physics-informed sensor model with an implicit neural geometry and reflectance model to directly synthesize raw radar measurements. We validate the effectiveness of the method across diverse outdoor scenarios, including urban scenes with dense vehicles and infrastructure.
  arXiv  Detail & Related papers  (2024-05-07T20:44:48Z)
• Diffusion-Based Point Cloud Super-Resolution for mmWave Radar Data [8.552647576661174]
  millimeter-wave radar sensor maintains stable performance under adverse environmental conditions. Radar point clouds are relatively sparse and contain massive ghost points. We propose a novel point cloud super-resolution approach for 3D mmWave radar data, named Radar-diffusion.
  arXiv  Detail & Related papers  (2024-04-09T04:41:05Z)
• Timely Fusion of Surround Radar/Lidar for Object Detection in Autonomous Driving Systems [13.998883144668941]
  Fusing Radar and Lidar sensor data can fully utilize their complementary advantages and provide more accurate reconstruction of the surrounding. Existing Radar/Lidar fusion methods have to work at the low frequency of surround Radar. This paper develops techniques to fuse surround Radar/Lidar with working frequency only limited by the faster surround Lidar.
  arXiv  Detail & Related papers  (2023-09-09T14:22:12Z)
• ROFusion: Efficient Object Detection using Hybrid Point-wise Radar-Optical Fusion [14.419658061805507]
  We propose a hybrid point-wise Radar-Optical fusion approach for object detection in autonomous driving scenarios. The framework benefits from dense contextual information from both the range-doppler spectrum and images which are integrated to learn a multi-modal feature representation.
  arXiv  Detail & Related papers  (2023-07-17T04:25:46Z)
• Bi-LRFusion: Bi-Directional LiDAR-Radar Fusion for 3D Dynamic Object Detection [78.59426158981108]
  We introduce a bi-directional LiDAR-Radar fusion framework, termed Bi-LRFusion, to tackle the challenges and improve 3D detection for dynamic objects. We conduct extensive experiments on nuScenes and ORR datasets, and show that our Bi-LRFusion achieves state-of-the-art performance for detecting dynamic objects.
  arXiv  Detail & Related papers  (2023-06-02T10:57:41Z)
• Semantic Segmentation of Radar Detections using Convolutions on Point Clouds [59.45414406974091]
  We introduce a deep-learning based method to convolve radar detections into point clouds. We adapt this algorithm to radar-specific properties through distance-dependent clustering and pre-processing of input point clouds. Our network outperforms state-of-the-art approaches that are based on PointNet++ on the task of semantic segmentation of radar point clouds.
  arXiv  Detail & Related papers  (2023-05-22T07:09:35Z)
• LiRaNet: End-to-End Trajectory Prediction using Spatio-Temporal Radar Fusion [52.59664614744447]
  We present LiRaNet, a novel end-to-end trajectory prediction method which utilizes radar sensor information along with widely used lidar and high definition (HD) maps. automotive radar provides rich, complementary information, allowing for longer range vehicle detection as well as instantaneous velocity measurements.
  arXiv  Detail & Related papers  (2020-10-02T00:13:00Z)
• RadarNet: Exploiting Radar for Robust Perception of Dynamic Objects [73.80316195652493]
  We tackle the problem of exploiting Radar for perception in the context of self-driving cars. We propose a new solution that exploits both LiDAR and Radar sensors for perception. Our approach, dubbed RadarNet, features a voxel-based early fusion and an attention-based late fusion.
  arXiv  Detail & Related papers  (2020-07-28T17:15:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.