Vision-RADAR fusion for Robotics BEV Detections: A Survey
- URL: http://arxiv.org/abs/2302.06643v1
- Date: Mon, 13 Feb 2023 19:10:44 GMT
- Title: Vision-RADAR fusion for Robotics BEV Detections: A Survey
- Authors: Apoorv Singh
- Abstract summary: Survey on Vision-Radar fusion approaches for a BEV object detection system.
We go through the background information viz., object detection tasks, choice of sensors, sensor setup, benchmark datasets and evaluation metrics for a robotic perception system.
We propose possible future trends for vision-radar fusion to enlighten future research.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Due to the trending need of building autonomous robotic perception system,
sensor fusion has attracted a lot of attention amongst researchers and
engineers to make best use of cross-modality information. However, in order to
build a robotic platform at scale we need to emphasize on autonomous robot
platform bring-up cost as well. Cameras and radars, which inherently includes
complementary perception information, has potential for developing autonomous
robotic platform at scale. However, there is a limited work around radar fused
with Vision, compared to LiDAR fused with vision work. In this paper, we tackle
this gap with a survey on Vision-Radar fusion approaches for a BEV object
detection system. First we go through the background information viz., object
detection tasks, choice of sensors, sensor setup, benchmark datasets and
evaluation metrics for a robotic perception system. Later, we cover
per-modality (Camera and RADAR) data representation, then we go into detail
about sensor fusion techniques based on sub-groups viz., early-fusion,
deep-fusion, and late-fusion to easily understand the pros and cons of each
method. Finally, we propose possible future trends for vision-radar fusion to
enlighten future research. Regularly updated summary can be found at:
https://github.com/ApoorvRoboticist/Vision-RADAR-Fusion-BEV-Survey
Related papers
- NeRF in Robotics: A Survey [95.11502610414803]
The recent emergence of neural implicit representations has introduced radical innovation to computer vision and robotics fields.
NeRF has sparked a trend because of the huge representational advantages, such as simplified mathematical models, compact environment storage, and continuous scene representations.
arXiv Detail & Related papers (2024-05-02T14:38:18Z) - Echoes Beyond Points: Unleashing the Power of Raw Radar Data in
Multi-modality Fusion [74.84019379368807]
We propose a novel method named EchoFusion to skip the existing radar signal processing pipeline.
Specifically, we first generate the Bird's Eye View (BEV) queries and then take corresponding spectrum features from radar to fuse with other sensors.
arXiv Detail & Related papers (2023-07-31T09:53:50Z) - ROFusion: Efficient Object Detection using Hybrid Point-wise
Radar-Optical Fusion [14.419658061805507]
We propose a hybrid point-wise Radar-Optical fusion approach for object detection in autonomous driving scenarios.
The framework benefits from dense contextual information from both the range-doppler spectrum and images which are integrated to learn a multi-modal feature representation.
arXiv Detail & Related papers (2023-07-17T04:25:46Z) - Radar-Camera Fusion for Object Detection and Semantic Segmentation in
Autonomous Driving: A Comprehensive Review [7.835577409160127]
This review focuses on perception tasks related to object detection and semantic segmentation.
In the review, we address interrogative questions, including "why to fuse", "what to fuse", "where to fuse", "when to fuse", and "how to fuse"
arXiv Detail & Related papers (2023-04-20T15:48:50Z) - Transformer-Based Sensor Fusion for Autonomous Driving: A Survey [0.0]
Transformers-based detection head and CNN-based feature encoder to extract features from raw sensor-data has emerged as one of the best performing sensor-fusion 3D-detection-framework.
We briefly go through the Vision transformers (ViT) basics, so that readers can easily follow through the paper.
In conclusion we summarize with sensor-fusion trends to follow and provoke future research.
arXiv Detail & Related papers (2023-02-22T16:28:20Z) - MmWave Radar and Vision Fusion based Object Detection for Autonomous
Driving: A Survey [15.316597644398188]
Millimeter wave (mmWave) radar and vision fusion is a mainstream solution for accurate obstacle detection.
This article presents a detailed survey on mmWave radar and vision fusion based obstacle detection methods.
arXiv Detail & Related papers (2021-08-06T08:38:42Z) - Multi-Modal 3D Object Detection in Autonomous Driving: a Survey [10.913958563906931]
Self-driving cars are equipped with a suite of sensors to conduct robust and accurate environment perception.
As the number and type of sensors keep increasing, combining them for better perception is becoming a natural trend.
This survey devotes to review recent fusion-based 3D detection deep learning models that leverage multiple sensor data sources.
arXiv Detail & Related papers (2021-06-24T02:52:12Z) - Domain and Modality Gaps for LiDAR-based Person Detection on Mobile
Robots [91.01747068273666]
This paper studies existing LiDAR-based person detectors with a particular focus on mobile robot scenarios.
Experiments revolve around the domain gap between driving and mobile robot scenarios, as well as the modality gap between 3D and 2D LiDAR sensors.
Results provide practical insights into LiDAR-based person detection and facilitate informed decisions for relevant mobile robot designs and applications.
arXiv Detail & Related papers (2021-06-21T16:35:49Z) - PC-DAN: Point Cloud based Deep Affinity Network for 3D Multi-Object
Tracking (Accepted as an extended abstract in JRDB-ACT Workshop at CVPR21) [68.12101204123422]
A point cloud is a dense compilation of spatial data in 3D coordinates.
We propose a PointNet-based approach for 3D Multi-Object Tracking (MOT)
arXiv Detail & Related papers (2021-06-03T05:36:39Z) - Rapid Exploration for Open-World Navigation with Latent Goal Models [78.45339342966196]
We describe a robotic learning system for autonomous exploration and navigation in diverse, open-world environments.
At the core of our method is a learned latent variable model of distances and actions, along with a non-parametric topological memory of images.
We use an information bottleneck to regularize the learned policy, giving us (i) a compact visual representation of goals, (ii) improved generalization capabilities, and (iii) a mechanism for sampling feasible goals for exploration.
arXiv Detail & Related papers (2021-04-12T23:14:41Z) - RadarNet: Exploiting Radar for Robust Perception of Dynamic Objects [73.80316195652493]
We tackle the problem of exploiting Radar for perception in the context of self-driving cars.
We propose a new solution that exploits both LiDAR and Radar sensors for perception.
Our approach, dubbed RadarNet, features a voxel-based early fusion and an attention-based late fusion.
arXiv Detail & Related papers (2020-07-28T17:15:02Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.