VIRUS-NeRF -- Vision, InfraRed and UltraSonic based Neural Radiance Fields
- URL: http://arxiv.org/abs/2403.09477v2
- Date: Wed, 14 Aug 2024 12:43:52 GMT
- Title: VIRUS-NeRF -- Vision, InfraRed and UltraSonic based Neural Radiance Fields
- Authors: Nicolaj Schmid, Cornelius von Einem, Cesar Cadena, Roland Siegwart, Lorenz Hruby, Florian Tschopp,
- Abstract summary: We propose to use cost-effective low-resolution ranging sensors, such as ultrasonic and infrared time-of-flight sensors.
VIRUS-NeRF incorporates depth measurements from ultrasonic and infrared sensors to update the occupancy grid used for ray marching.
- Score: 24.151647704246013
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Autonomous mobile robots are an increasingly integral part of modern factory and warehouse operations. Obstacle detection, avoidance and path planning are critical safety-relevant tasks, which are often solved using expensive LiDAR sensors and depth cameras. We propose to use cost-effective low-resolution ranging sensors, such as ultrasonic and infrared time-of-flight sensors by developing VIRUS-NeRF - Vision, InfraRed, and UltraSonic based Neural Radiance Fields. Building upon Instant Neural Graphics Primitives with a Multiresolution Hash Encoding (Instant-NGP), VIRUS-NeRF incorporates depth measurements from ultrasonic and infrared sensors and utilizes them to update the occupancy grid used for ray marching. Experimental evaluation in 2D demonstrates that VIRUS-NeRF achieves comparable mapping performance to LiDAR point clouds regarding coverage. Notably, in small environments, its accuracy aligns with that of LiDAR measurements, while in larger ones, it is bounded by the utilized ultrasonic sensors. An in-depth ablation study reveals that adding ultrasonic and infrared sensors is highly effective when dealing with sparse data and low view variation. Further, the proposed occupancy grid of VIRUS-NeRF improves the mapping capabilities and increases the training speed by 46% compared to Instant-NGP. Overall, VIRUS-NeRF presents a promising approach for cost-effective local mapping in mobile robotics, with potential applications in safety and navigation tasks. The code can be found at https://github.com/ethz-asl/virus nerf.
Related papers
- Radar Fields: Frequency-Space Neural Scene Representations for FMCW Radar [62.51065633674272]
We introduce Radar Fields - a neural scene reconstruction method designed for active radar imagers.
Our approach unites an explicit, physics-informed sensor model with an implicit neural geometry and reflectance model to directly synthesize raw radar measurements.
We validate the effectiveness of the method across diverse outdoor scenarios, including urban scenes with dense vehicles and infrastructure.
arXiv Detail & Related papers (2024-05-07T20:44:48Z) - Echoes Beyond Points: Unleashing the Power of Raw Radar Data in
Multi-modality Fusion [74.84019379368807]
We propose a novel method named EchoFusion to skip the existing radar signal processing pipeline.
Specifically, we first generate the Bird's Eye View (BEV) queries and then take corresponding spectrum features from radar to fuse with other sensors.
arXiv Detail & Related papers (2023-07-31T09:53:50Z) - DensePose From WiFi [86.61881052177228]
We develop a deep neural network that maps the phase and amplitude of WiFi signals to UV coordinates within 24 human regions.
Our model can estimate the dense pose of multiple subjects, with comparable performance to image-based approaches.
arXiv Detail & Related papers (2022-12-31T16:48:43Z) - NVRadarNet: Real-Time Radar Obstacle and Free Space Detection for
Autonomous Driving [57.03126447713602]
We present a deep neural network (DNN) that detects dynamic obstacles and drivable free space using automotive RADAR sensors.
The network runs faster than real time on an embedded GPU and shows good generalization across geographic regions.
arXiv Detail & Related papers (2022-09-29T01:30:34Z) - MDPose: Human Skeletal Motion Reconstruction Using WiFi Micro-Doppler
Signatures [4.92674421365689]
We propose MDPose, a novel framework for human skeletal motion reconstruction based on WiFi micro-Doppler signatures.
It provides an effective solution to track human activities by reconstructing a skeleton model with 17 key points.
MDPose outperforms state-of-the-art RF-based pose estimation systems.
arXiv Detail & Related papers (2022-01-11T21:46:28Z) - NeRF-SR: High-Quality Neural Radiance Fields using Super-Sampling [82.99453001445478]
We present NeRF-SR, a solution for high-resolution (HR) novel view synthesis with mostly low-resolution (LR) inputs.
Our method is built upon Neural Radiance Fields (NeRF) that predicts per-point density and color with a multi-layer perceptron.
arXiv Detail & Related papers (2021-12-03T07:33:47Z) - Complex-valued Convolutional Neural Networks for Enhanced Radar Signal
Denoising and Interference Mitigation [73.0103413636673]
We propose the use of Complex-Valued Convolutional Neural Networks (CVCNNs) to address the issue of mutual interference between radar sensors.
CVCNNs increase data efficiency, speeds up network training and substantially improves the conservation of phase information during interference removal.
arXiv Detail & Related papers (2021-04-29T10:06:29Z) - All-Weather Object Recognition Using Radar and Infrared Sensing [1.7513645771137178]
This thesis explores new sensing developments based on long wave polarised infrared (IR) imagery and imaging radar to recognise objects.
First, we developed a methodology based on Stokes parameters using polarised infrared data to recognise vehicles using deep neural networks.
Second, we explored the potential of using only the power spectrum captured by low-THz radar sensors to perform object recognition in a controlled scenario.
Last, we created a new large-scale dataset in the "wild" with many different weather scenarios showing radar robustness to detect vehicles in adverse weather.
arXiv Detail & Related papers (2020-10-30T14:16:39Z) - Super Low Resolution RF Powered Accelerometers for Alerting on
Hospitalized Patient Bed Exits [3.2654923574107357]
Falls have serious consequences and are prevalent in acute hospitals and nursing homes caring for older people. Technological interventions to mitigate the risk of falling aim to automatically monitor bed-exit events and alert healthcare personnel to provide timely supervisions.
We observe that frequency-domain information related to patient activities exist predominantly in very low frequencies.
We investigate a batteryless sensing modality with low cost wirelessly powered Radio Frequency Identification (RFID) technology with the potential for convenient integration into clothing, such as hospital gowns.
arXiv Detail & Related papers (2020-03-19T00:58:30Z) - Harvesting Ambient RF for Presence Detection Through Deep Learning [12.535149305258171]
This paper explores the use of ambient radio frequency (RF) signals for human presence detection through deep learning.
Using WiFi signal as an example, we demonstrate that the channel state information (CSI) obtained at the receiver contains rich information about the propagation environment.
A convolutional neural network (CNN) properly trained with both magnitude and phase information is then designed to achieve reliable presence detection.
arXiv Detail & Related papers (2020-02-13T20:35:55Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.