ISETAuto: Detecting vehicles with depth and radiance information
- URL: http://arxiv.org/abs/2101.01843v2
- Date: Thu, 7 Jan 2021 02:25:02 GMT
- Title: ISETAuto: Detecting vehicles with depth and radiance information
- Authors: Zhenyi Liu, Joyce Farrell, Brian Wandell
- Abstract summary: We compare the performance of a ResNet for vehicle detection in complex, daytime, driving scenes.
For a hybrid system that combines a depth map and radiance image, the average precision is higher than using depth or radiance alone.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Autonomous driving applications use two types of sensor systems to identify
vehicles - depth sensing LiDAR and radiance sensing cameras. We compare the
performance (average precision) of a ResNet for vehicle detection in complex,
daytime, driving scenes when the input is a depth map (D = d(x,y)), a radiance
image (L = r(x,y)), or both [D,L]. (1) When the spatial sampling resolution of
the depth map and radiance image are equal to typical camera resolutions, a
ResNet detects vehicles at higher average precision from depth than radiance.
(2) As the spatial sampling of the depth map declines to the range of current
LiDAR devices, the ResNet average precision is higher for radiance than depth.
(3) For a hybrid system that combines a depth map and radiance image, the
average precision is higher than using depth or radiance alone. We established
these observations in simulation and then confirmed them using realworld data.
The advantage of combining depth and radiance can be explained by noting that
the two type of information have complementary weaknesses. The radiance data
are limited by dynamic range and motion blur. The LiDAR data have relatively
low spatial resolution. The ResNet combines the two data sources effectively to
improve overall vehicle detection.
Related papers
- Redefining Automotive Radar Imaging: A Domain-Informed 1D Deep Learning Approach for High-Resolution and Efficient Performance [6.784861785632841]
Our study redefines radar imaging super-resolution as a one-dimensional (1D) signal super-resolution spectra estimation problem.
Our tailored deep learning network for automotive radar imaging exhibits remarkable scalability, parameter efficiency and fast inference speed.
Our SR-SPECNet sets a new benchmark in producing high-resolution radar range-azimuth images.
arXiv Detail & Related papers (2024-06-11T16:07:08Z) - Robust Depth Enhancement via Polarization Prompt Fusion Tuning [112.88371907047396]
We present a framework that leverages polarization imaging to improve inaccurate depth measurements from various depth sensors.
Our method first adopts a learning-based strategy where a neural network is trained to estimate a dense and complete depth map from polarization data and a sensor depth map from different sensors.
To further improve the performance, we propose a Polarization Prompt Fusion Tuning (PPFT) strategy to effectively utilize RGB-based models pre-trained on large-scale datasets.
arXiv Detail & Related papers (2024-04-05T17:55:33Z) - Neural Rendering based Urban Scene Reconstruction for Autonomous Driving [8.007494499012624]
We propose a multimodal 3D scene reconstruction using a framework combining neural implicit surfaces and radiance fields.
Dense 3D reconstruction has many applications in automated driving including automated annotation validation.
We demonstrate qualitative and quantitative results on challenging automotive scenes.
arXiv Detail & Related papers (2024-02-09T23:20:23Z) - R-C-P Method: An Autonomous Volume Calculation Method Using Image
Processing and Machine Vision [0.0]
Two cameras were used to measure the dimensions of a rectangular object in real-time.
The R-C-P method is developed using image processing and edge detection.
In addition to the surface areas, the R-C-P method also detects discontinuous edges or volumes.
arXiv Detail & Related papers (2023-08-19T15:39:27Z) - LiDAR-as-Camera for End-to-End Driving [1.0323063834827415]
Ouster LiDARs can output surround-view LiDAR-images with depth, intensity, and ambient radiation channels.
These measurements originate from the same sensor, rendering them perfectly aligned in time and space.
We demonstrate that such LiDAR-images are sufficient for the real-car road-following task and perform at least equally to camera-based models in the tested conditions.
In the second direction of study, we reveal that the temporal smoothness of off-policy prediction sequences correlates equally well with actual on-policy driving ability as the commonly used mean absolute error.
arXiv Detail & Related papers (2022-06-30T10:06:49Z) - Joint Learning of Salient Object Detection, Depth Estimation and Contour
Extraction [91.43066633305662]
We propose a novel multi-task and multi-modal filtered transformer (MMFT) network for RGB-D salient object detection (SOD)
Specifically, we unify three complementary tasks: depth estimation, salient object detection and contour estimation. The multi-task mechanism promotes the model to learn the task-aware features from the auxiliary tasks.
Experiments show that it not only significantly surpasses the depth-based RGB-D SOD methods on multiple datasets, but also precisely predicts a high-quality depth map and salient contour at the same time.
arXiv Detail & Related papers (2022-03-09T17:20:18Z) - Towards Fast and Accurate Real-World Depth Super-Resolution: Benchmark
Dataset and Baseline [48.69396457721544]
We build a large-scale dataset named "RGB-D-D" to promote the study of depth map super-resolution (SR)
We provide a fast depth map super-resolution (FDSR) baseline, in which the high-frequency component adaptively decomposed from RGB image to guide the depth map SR.
For the real-world LR depth maps, our algorithm can produce more accurate HR depth maps with clearer boundaries and to some extent correct the depth value errors.
arXiv Detail & Related papers (2021-04-13T13:27:26Z) - Sparse LiDAR and Stereo Fusion (SLS-Fusion) for Depth Estimationand 3D
Object Detection [3.5488685789514736]
SLS-Fusion is a new approach to fuse data from 4-beam LiDAR and a stereo camera via a neural network for depth estimation.
Since 4-beam LiDAR is cheaper than the well-known 64-beam LiDAR, this approach is also classified as a low-cost sensors-based method.
arXiv Detail & Related papers (2021-03-05T23:10:09Z) - PLUME: Efficient 3D Object Detection from Stereo Images [95.31278688164646]
Existing methods tackle the problem in two steps: first depth estimation is performed, a pseudo LiDAR point cloud representation is computed from the depth estimates, and then object detection is performed in 3D space.
We propose a model that unifies these two tasks in the same metric space.
Our approach achieves state-of-the-art performance on the challenging KITTI benchmark, with significantly reduced inference time compared with existing methods.
arXiv Detail & Related papers (2021-01-17T05:11:38Z) - RadarNet: Exploiting Radar for Robust Perception of Dynamic Objects [73.80316195652493]
We tackle the problem of exploiting Radar for perception in the context of self-driving cars.
We propose a new solution that exploits both LiDAR and Radar sensors for perception.
Our approach, dubbed RadarNet, features a voxel-based early fusion and an attention-based late fusion.
arXiv Detail & Related papers (2020-07-28T17:15:02Z) - Drone-based RGB-Infrared Cross-Modality Vehicle Detection via
Uncertainty-Aware Learning [59.19469551774703]
Drone-based vehicle detection aims at finding the vehicle locations and categories in an aerial image.
We construct a large-scale drone-based RGB-Infrared vehicle detection dataset, termed DroneVehicle.
Our DroneVehicle collects 28, 439 RGB-Infrared image pairs, covering urban roads, residential areas, parking lots, and other scenarios from day to night.
arXiv Detail & Related papers (2020-03-05T05:29:44Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.