Self-Supervised Scene Flow Estimation with 4D Automotive Radar
- URL: http://arxiv.org/abs/2203.01137v1
- Date: Wed, 2 Mar 2022 14:28:12 GMT
- Title: Self-Supervised Scene Flow Estimation with 4D Automotive Radar
- Authors: Fangqiang Ding, Zhijun Pan, Yimin Deng, Jianning Deng, Chris Xiaoxuan
Lu
- Abstract summary: It remains largely unknown how to estimate the scene flow from a 4D radar.
Compared with the LiDAR point clouds, radar data are drastically sparser, noisier and in much lower resolution.
This work aims to address the above challenges and estimate scene flow from 4D radar point clouds by leveraging self-supervised learning.
- Score: 7.3287286038702035
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Scene flow allows autonomous vehicles to reason about the arbitrary motion of
multiple independent objects which is the key to long-term mobile autonomy.
While estimating the scene flow from LiDAR has progressed recently, it remains
largely unknown how to estimate the scene flow from a 4D radar - an
increasingly popular automotive sensor for its robustness against adverse
weather and lighting conditions. Compared with the LiDAR point clouds, radar
data are drastically sparser, noisier and in much lower resolution. Annotated
datasets for radar scene flow are also in absence and costly to acquire in the
real world. These factors jointly pose the radar scene flow estimation as a
challenging problem. This work aims to address the above challenges and
estimate scene flow from 4D radar point clouds by leveraging self-supervised
learning. A robust scene flow estimation architecture and three novel losses
are bespoken designed to cope with intractable radar data. Real-world
experimental results validate that our method is able to robustly estimate the
radar scene flow in the wild and effectively supports the downstream task of
motion segmentation.
Related papers
- RadarOcc: Robust 3D Occupancy Prediction with 4D Imaging Radar [15.776076554141687]
3D occupancy-based perception pipeline has significantly advanced autonomous driving.
Current methods rely on LiDAR or camera inputs for 3D occupancy prediction.
We introduce a novel approach that utilizes 4D imaging radar sensors for 3D occupancy prediction.
arXiv Detail & Related papers (2024-05-22T21:48:17Z) - Radar Fields: Frequency-Space Neural Scene Representations for FMCW Radar [62.51065633674272]
We introduce Radar Fields - a neural scene reconstruction method designed for active radar imagers.
Our approach unites an explicit, physics-informed sensor model with an implicit neural geometry and reflectance model to directly synthesize raw radar measurements.
We validate the effectiveness of the method across diverse outdoor scenarios, including urban scenes with dense vehicles and infrastructure.
arXiv Detail & Related papers (2024-05-07T20:44:48Z) - Bootstrapping Autonomous Driving Radars with Self-Supervised Learning [13.13679517730015]
Training radar models is hindered by the cost and difficulty of annotating large-scale radar data.
We propose a self-supervised learning framework to leverage the large amount of unlabeled radar data to pre-train radar-only embeddings for self-driving perception tasks.
When used for downstream object detection, we demonstrate that the proposed self-supervision framework can improve the accuracy of state-of-the-art supervised baselines by $5.8%$ in mAP.
arXiv Detail & Related papers (2023-12-07T18:38:39Z) - MVFAN: Multi-View Feature Assisted Network for 4D Radar Object Detection [15.925365473140479]
4D radar is recognized for its resilience and cost-effectiveness under adverse weather conditions.
Unlike LiDAR and cameras, radar remains unimpaired by harsh weather conditions.
We propose a framework for developing radar-based 3D object detection for autonomous vehicles.
arXiv Detail & Related papers (2023-10-25T06:10:07Z) - Radars for Autonomous Driving: A Review of Deep Learning Methods and
Challenges [0.021665899581403605]
Radar is a key component of the suite of perception sensors used for autonomous vehicles.
It is characterized by low resolution, sparsity, clutter, high uncertainty, and lack of good datasets.
Current radar models are often influenced by lidar and vision models, which are focused on optical features that are relatively weak in radar data.
arXiv Detail & Related papers (2023-06-15T17:37:52Z) - Vision in adverse weather: Augmentation using CycleGANs with various
object detectors for robust perception in autonomous racing [70.16043883381677]
In autonomous racing, the weather can change abruptly, causing significant degradation in perception, resulting in ineffective manoeuvres.
In order to improve detection in adverse weather, deep-learning-based models typically require extensive datasets captured in such conditions.
We introduce an approach of using synthesised adverse condition datasets in autonomous racing (generated using CycleGAN) to improve the performance of four out of five state-of-the-art detectors.
arXiv Detail & Related papers (2022-01-10T10:02:40Z) - R4Dyn: Exploring Radar for Self-Supervised Monocular Depth Estimation of
Dynamic Scenes [69.6715406227469]
Self-supervised monocular depth estimation in driving scenarios has achieved comparable performance to supervised approaches.
We present R4Dyn, a novel set of techniques to use cost-efficient radar data on top of a self-supervised depth estimation framework.
arXiv Detail & Related papers (2021-08-10T17:57:03Z) - Lidar Light Scattering Augmentation (LISA): Physics-based Simulation of
Adverse Weather Conditions for 3D Object Detection [60.89616629421904]
Lidar-based object detectors are critical parts of the 3D perception pipeline in autonomous navigation systems such as self-driving cars.
They are sensitive to adverse weather conditions such as rain, snow and fog due to reduced signal-to-noise ratio (SNR) and signal-to-background ratio (SBR)
arXiv Detail & Related papers (2021-07-14T21:10:47Z) - LiRaNet: End-to-End Trajectory Prediction using Spatio-Temporal Radar
Fusion [52.59664614744447]
We present LiRaNet, a novel end-to-end trajectory prediction method which utilizes radar sensor information along with widely used lidar and high definition (HD) maps.
automotive radar provides rich, complementary information, allowing for longer range vehicle detection as well as instantaneous velocity measurements.
arXiv Detail & Related papers (2020-10-02T00:13:00Z) - RadarNet: Exploiting Radar for Robust Perception of Dynamic Objects [73.80316195652493]
We tackle the problem of exploiting Radar for perception in the context of self-driving cars.
We propose a new solution that exploits both LiDAR and Radar sensors for perception.
Our approach, dubbed RadarNet, features a voxel-based early fusion and an attention-based late fusion.
arXiv Detail & Related papers (2020-07-28T17:15:02Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.