Bootstrapping Autonomous Driving Radars with Self-Supervised Learning

• URL: http://arxiv.org/abs/2312.04519v3
• Date: Thu, 18 Apr 2024 10:40:35 GMT
• Title: Bootstrapping Autonomous Driving Radars with Self-Supervised Learning
• Authors: Yiduo Hao, Sohrab Madani, Junfeng Guan, Mohammed Alloulah, Saurabh Gupta, Haitham Hassanieh,
• Abstract summary: Training radar models is hindered by the cost and difficulty of annotating large-scale radar data. We propose a self-supervised learning framework to leverage the large amount of unlabeled radar data to pre-train radar-only embeddings for self-driving perception tasks. When used for downstream object detection, we demonstrate that the proposed self-supervision framework can improve the accuracy of state-of-the-art supervised baselines by $5.8%$ in mAP.
• Score: 13.13679517730015
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The perception of autonomous vehicles using radars has attracted increased research interest due its ability to operate in fog and bad weather. However, training radar models is hindered by the cost and difficulty of annotating large-scale radar data. To overcome this bottleneck, we propose a self-supervised learning framework to leverage the large amount of unlabeled radar data to pre-train radar-only embeddings for self-driving perception tasks. The proposed method combines radar-to-radar and radar-to-vision contrastive losses to learn a general representation from unlabeled radar heatmaps paired with their corresponding camera images. When used for downstream object detection, we demonstrate that the proposed self-supervision framework can improve the accuracy of state-of-the-art supervised baselines by $5.8\%$ in mAP. Code is available at \url{https://github.com/yiduohao/Radical}.

Related papers

• Radar Fields: Frequency-Space Neural Scene Representations for FMCW Radar [62.51065633674272]
  We introduce Radar Fields - a neural scene reconstruction method designed for active radar imagers. Our approach unites an explicit, physics-informed sensor model with an implicit neural geometry and reflectance model to directly synthesize raw radar measurements. We validate the effectiveness of the method across diverse outdoor scenarios, including urban scenes with dense vehicles and infrastructure.
  arXiv  Detail & Related papers  (2024-05-07T20:44:48Z)
• Echoes Beyond Points: Unleashing the Power of Raw Radar Data in Multi-modality Fusion [74.84019379368807]
  We propose a novel method named EchoFusion to skip the existing radar signal processing pipeline. Specifically, we first generate the Bird's Eye View (BEV) queries and then take corresponding spectrum features from radar to fuse with other sensors.
  arXiv  Detail & Related papers  (2023-07-31T09:53:50Z)
• RadarFormer: Lightweight and Accurate Real-Time Radar Object Detection Model [13.214257841152033]
  Radar-centric data sets do not get a lot of attention in the development of deep learning techniques for radar perception. We propose a transformers-based model, named RadarFormer, that utilizes state-of-the-art developments in vision deep learning. Our model also introduces a channel-chirp-time merging module that reduces the size and complexity of our models by more than 10 times without compromising accuracy.
  arXiv  Detail & Related papers  (2023-04-17T17:07:35Z)
• NVRadarNet: Real-Time Radar Obstacle and Free Space Detection for Autonomous Driving [57.03126447713602]
  We present a deep neural network (DNN) that detects dynamic obstacles and drivable free space using automotive RADAR sensors. The network runs faster than real time on an embedded GPU and shows good generalization across geographic regions.
  arXiv  Detail & Related papers  (2022-09-29T01:30:34Z)
• Exploiting Temporal Relations on Radar Perception for Autonomous Driving [26.736501544682294]
  We exploit the temporal information from successive ego-centric bird-eye-view radar image frames for radar object recognition. We propose a temporal relational layer to explicitly model the relations between objects within successive radar images.
  arXiv  Detail & Related papers  (2022-04-03T23:52:25Z)
• Deep Instance Segmentation with High-Resolution Automotive Radar [2.167586397005864]
  We propose two efficient methods for instance segmentation with radar detection points. One is implemented in an end-to-end deep learning driven fashion using PointNet++ framework. The other is based on clustering of the radar detection points with semantic information.
  arXiv  Detail & Related papers  (2021-10-05T01:18:27Z)
• R4Dyn: Exploring Radar for Self-Supervised Monocular Depth Estimation of Dynamic Scenes [69.6715406227469]
  Self-supervised monocular depth estimation in driving scenarios has achieved comparable performance to supervised approaches. We present R4Dyn, a novel set of techniques to use cost-efficient radar data on top of a self-supervised depth estimation framework.
  arXiv  Detail & Related papers  (2021-08-10T17:57:03Z)
• Radar Artifact Labeling Framework (RALF): Method for Plausible Radar Detections in Datasets [2.5899040911480187]
  We propose a cross sensor Radar Artifact Labeling Framework (RALF) for labeling sparse radar point clouds. RALF provides plausibility labels for radar raw detections, distinguishing between artifacts and targets. We validate the results by evaluating error metrics on semi-manually labeled ground truth dataset of $3.28cdot106$ points.
  arXiv  Detail & Related papers  (2020-12-03T15:11:31Z)
• LiRaNet: End-to-End Trajectory Prediction using Spatio-Temporal Radar Fusion [52.59664614744447]
  We present LiRaNet, a novel end-to-end trajectory prediction method which utilizes radar sensor information along with widely used lidar and high definition (HD) maps. automotive radar provides rich, complementary information, allowing for longer range vehicle detection as well as instantaneous velocity measurements.
  arXiv  Detail & Related papers  (2020-10-02T00:13:00Z)
• RadarNet: Exploiting Radar for Robust Perception of Dynamic Objects [73.80316195652493]
  We tackle the problem of exploiting Radar for perception in the context of self-driving cars. We propose a new solution that exploits both LiDAR and Radar sensors for perception. Our approach, dubbed RadarNet, features a voxel-based early fusion and an attention-based late fusion.
  arXiv  Detail & Related papers  (2020-07-28T17:15:02Z)
• Probabilistic Oriented Object Detection in Automotive Radar [8.281391209717103]
  We propose a deep-learning based algorithm for radar object detection. We created a new multimodal dataset with 102544 frames of raw radar and synchronized LiDAR data. Our best performing radar detection model achieves 77.28% AP under oriented IoU of 0.3.
  arXiv  Detail & Related papers  (2020-04-11T05:29:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.