TEDNet: Twin Encoder Decoder Neural Network for 2D Camera and LiDAR Road Detection

• URL: http://arxiv.org/abs/2405.08429v1
• Date: Tue, 14 May 2024 08:45:34 GMT
• Title: TEDNet: Twin Encoder Decoder Neural Network for 2D Camera and LiDAR Road Detection
• Authors: Martín Bayón-Gutiérrez, María Teresa García-Ordás, Héctor Alaiz Moretón, Jose Aveleira-Mata, Sergio Rubio Martín, José Alberto Benítez-Andrades,
• Abstract summary: A novel Convolutional Neural Network model is proposed for the accurate estimation of the roadway surface. Our model is based on the use of a Twin-Decoder Neural Network (TEDNet) for independent camera and LiDAR feature extraction. Bird's Eye View projections of the camera and LiDAR data are used in this model to perform semantic segmentation on whether each pixel belongs to the road surface.
• Score: 2.8038082486377114
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Robust road surface estimation is required for autonomous ground vehicles to navigate safely. Despite it becoming one of the main targets for autonomous mobility researchers in recent years, it is still an open problem in which cameras and LiDAR sensors have demonstrated to be adequate to predict the position, size and shape of the road a vehicle is driving on in different environments. In this work, a novel Convolutional Neural Network model is proposed for the accurate estimation of the roadway surface. Furthermore, an ablation study has been conducted to investigate how different encoding strategies affect model performance, testing 6 slightly different neural network architectures. Our model is based on the use of a Twin Encoder-Decoder Neural Network (TEDNet) for independent camera and LiDAR feature extraction, and has been trained and evaluated on the Kitti-Road dataset. Bird's Eye View projections of the camera and LiDAR data are used in this model to perform semantic segmentation on whether each pixel belongs to the road surface. The proposed method performs among other state-of-the-art methods and operates at the same frame-rate as the LiDAR and cameras, so it is adequate for its use in real-time applications.

Related papers

• Multi-Object Tracking with Camera-LiDAR Fusion for Autonomous Driving [0.764971671709743]
  The proposed MOT algorithm comprises a three-step association process, an Extended Kalman filter for estimating the motion of each detected dynamic obstacle, and a track management phase. Unlike most state-of-the-art multi-modal MOT approaches, the proposed algorithm does not rely on maps or knowledge of the ego global pose. The algorithm is validated both in simulation and with real-world data, with satisfactory results.
  arXiv  Detail & Related papers  (2024-03-06T23:49:16Z)
• OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environments [77.0399450848749]
  We propose an OccNeRF method for training occupancy networks without 3D supervision. We parameterize the reconstructed occupancy fields and reorganize the sampling strategy to align with the cameras' infinite perceptive range. For semantic occupancy prediction, we design several strategies to polish the prompts and filter the outputs of a pretrained open-vocabulary 2D segmentation model.
  arXiv  Detail & Related papers  (2023-12-14T18:58:52Z)
• UnLoc: A Universal Localization Method for Autonomous Vehicles using LiDAR, Radar and/or Camera Input [51.150605800173366]
  UnLoc is a novel unified neural modeling approach for localization with multi-sensor input in all weather conditions. Our method is extensively evaluated on Oxford Radar RobotCar, ApolloSouthBay and Perth-WA datasets.
  arXiv  Detail & Related papers  (2023-07-03T04:10:55Z)
• NVRadarNet: Real-Time Radar Obstacle and Free Space Detection for Autonomous Driving [57.03126447713602]
  We present a deep neural network (DNN) that detects dynamic obstacles and drivable free space using automotive RADAR sensors. The network runs faster than real time on an embedded GPU and shows good generalization across geographic regions.
  arXiv  Detail & Related papers  (2022-09-29T01:30:34Z)
• Benchmarking the Robustness of LiDAR-Camera Fusion for 3D Object Detection [58.81316192862618]
  Two critical sensors for 3D perception in autonomous driving are the camera and the LiDAR. fusing these two modalities can significantly boost the performance of 3D perception models. We benchmark the state-of-the-art fusion methods for the first time.
  arXiv  Detail & Related papers  (2022-05-30T09:35:37Z)
• Cross-Camera Trajectories Help Person Retrieval in a Camera Network [124.65912458467643]
  Existing methods often rely on purely visual matching or consider temporal constraints but ignore the spatial information of the camera network. We propose a pedestrian retrieval framework based on cross-camera generation, which integrates both temporal and spatial information. To verify the effectiveness of our method, we construct the first cross-camera pedestrian trajectory dataset.
  arXiv  Detail & Related papers  (2022-04-27T13:10:48Z)
• Monocular Vision-based Prediction of Cut-in Maneuvers with LSTM Networks [0.0]
  This study proposes a method to predict potentially dangerous cut-in maneuvers happening in the ego lane. We follow a computer vision-based approach that only employs a single in-vehicle RGB camera. Our algorithm consists of a CNN-based vehicle detection and tracking step and an LSTM-based maneuver classification step.
  arXiv  Detail & Related papers  (2022-03-21T02:30:36Z)
• Automatic Extraction of Road Networks from Satellite Images by using Adaptive Structural Deep Belief Network [0.0]
  Our model is applied to an automatic recognition method of road network system, called RoadTracer. RoadTracer can generate a road map on the ground surface from aerial photograph data. In order to improve the accuracy and the calculation time, our Adaptive DBN was implemented on the RoadTracer instead of the CNN.
  arXiv  Detail & Related papers  (2021-10-25T07:06:10Z)
• CNN-based Omnidirectional Object Detection for HermesBot Autonomous Delivery Robot with Preliminary Frame Classification [53.56290185900837]
  We propose an algorithm for optimizing a neural network for object detection using preliminary binary frame classification. An autonomous mobile robot with 6 rolling-shutter cameras on the perimeter providing a 360-degree field of view was used as the experimental setup.
  arXiv  Detail & Related papers  (2021-10-22T15:05:37Z)
• Pseudo-LiDAR Based Road Detection [5.9106199000537645]
  We propose a novel road detection approach with RGB being the only input during inference. We exploit pseudo-LiDAR using depth estimation, and propose a feature fusion network where RGB and learned depth information are fused. The proposed method achieves state-of-the-art performance on two challenging benchmarks, KITTI and R2D.
  arXiv  Detail & Related papers  (2021-07-28T11:21:42Z)
• 2nd Place Solution for Waymo Open Dataset Challenge - Real-time 2D Object Detection [26.086623067939605]
  In this report, we introduce a real-time method to detect the 2D objects from images. We leverage accelerationRT to optimize the inference time of our detection pipeline. Our framework achieves the latency of 45.8ms/frame on an Nvidia Tesla V100 GPU.
  arXiv  Detail & Related papers  (2021-06-16T11:32:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.