A Pedestrian Detection and Tracking Framework for Autonomous Cars: Efficient Fusion of Camera and LiDAR Data

• URL: http://arxiv.org/abs/2108.12375v1
• Date: Fri, 27 Aug 2021 16:16:01 GMT
• Title: A Pedestrian Detection and Tracking Framework for Autonomous Cars: Efficient Fusion of Camera and LiDAR Data
• Authors: Muhammad Mobaidul Islam, Abdullah Al Redwan Newaz, and Ali Karimoddini
• Abstract summary: This paper presents a novel method for pedestrian detection and tracking by fusing camera and LiDAR sensor data. The detection phase is performed by converting LiDAR streams to computationally tractable depth images, and then, a deep neural network is developed to identify pedestrian candidates. The tracking phase is a combination of the Kalman filter prediction and an optical flow algorithm to track multiple pedestrians in a scene.
• Score: 0.17205106391379021
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This paper presents a novel method for pedestrian detection and tracking by fusing camera and LiDAR sensor data. To deal with the challenges associated with the autonomous driving scenarios, an integrated tracking and detection framework is proposed. The detection phase is performed by converting LiDAR streams to computationally tractable depth images, and then, a deep neural network is developed to identify pedestrian candidates both in RGB and depth images. To provide accurate information, the detection phase is further enhanced by fusing multi-modal sensor information using the Kalman filter. The tracking phase is a combination of the Kalman filter prediction and an optical flow algorithm to track multiple pedestrians in a scene. We evaluate our framework on a real public driving dataset. Experimental results demonstrate that the proposed method achieves significant performance improvement over a baseline method that solely uses image-based pedestrian detection.

Related papers

• Multi-Object Tracking based on Imaging Radar 3D Object Detection [0.13499500088995461]
  This paper presents an approach for tracking surrounding traffic participants with a classical tracking algorithm. Learning based object detectors have been shown to work adequately on lidar and camera data, while learning based object detectors using standard radar data input have proven to be inferior. With the improvements to radar sensor technology in the form of imaging radars, the object detection performance on radar was greatly improved but is still limited compared to lidar sensors due to the sparsity of the radar point cloud. The tracking algorithm must overcome the limited detection quality while generating consistent tracks.
  arXiv  Detail & Related papers  (2024-06-03T05:46:23Z)
• Multi-Object Tracking with Camera-LiDAR Fusion for Autonomous Driving [0.764971671709743]
  The proposed MOT algorithm comprises a three-step association process, an Extended Kalman filter for estimating the motion of each detected dynamic obstacle, and a track management phase. Unlike most state-of-the-art multi-modal MOT approaches, the proposed algorithm does not rely on maps or knowledge of the ego global pose. The algorithm is validated both in simulation and with real-world data, with satisfactory results.
  arXiv  Detail & Related papers  (2024-03-06T23:49:16Z)
• Unsupervised Domain Adaptation for Self-Driving from Past Traversal Features [69.47588461101925]
  We propose a method to adapt 3D object detectors to new driving environments. Our approach enhances LiDAR-based detection models using spatial quantized historical features. Experiments on real-world datasets demonstrate significant improvements.
  arXiv  Detail & Related papers  (2023-09-21T15:00:31Z)
• UnLoc: A Universal Localization Method for Autonomous Vehicles using LiDAR, Radar and/or Camera Input [51.150605800173366]
  UnLoc is a novel unified neural modeling approach for localization with multi-sensor input in all weather conditions. Our method is extensively evaluated on Oxford Radar RobotCar, ApolloSouthBay and Perth-WA datasets.
  arXiv  Detail & Related papers  (2023-07-03T04:10:55Z)
• Trajectory Poisson multi-Bernoulli mixture filter for traffic monitoring using a drone [17.636403357588897]
  This paper proposes a multi-object tracking (MOT) algorithm for traffic monitoring using a drone equipped with optical and thermal cameras. Object detections on the images are obtained using a neural network for each type of camera. We have tested the accuracy of the resulting TPMBM filter in synthetic and experimental data sets.
  arXiv  Detail & Related papers  (2023-06-29T12:22:47Z)
• Federated Deep Learning Meets Autonomous Vehicle Perception: Design and Verification [168.67190934250868]
  Federated learning empowered connected autonomous vehicle (FLCAV) has been proposed. FLCAV preserves privacy while reducing communication and annotation costs. It is challenging to determine the network resources and road sensor poses for multi-stage training.
  arXiv  Detail & Related papers  (2022-06-03T23:55:45Z)
• Cross-Camera Trajectories Help Person Retrieval in a Camera Network [124.65912458467643]
  Existing methods often rely on purely visual matching or consider temporal constraints but ignore the spatial information of the camera network. We propose a pedestrian retrieval framework based on cross-camera generation, which integrates both temporal and spatial information. To verify the effectiveness of our method, we construct the first cross-camera pedestrian trajectory dataset.
  arXiv  Detail & Related papers  (2022-04-27T13:10:48Z)
• CFTrack: Center-based Radar and Camera Fusion for 3D Multi-Object Tracking [9.62721286522053]
  We propose an end-to-end network for joint object detection and tracking based on radar and camera sensor fusion. Our proposed method uses a center-based radar-camera fusion algorithm for object detection and utilizes a greedy algorithm for object association. We evaluate our method on the challenging nuScenes dataset, where it achieves 20.0 AMOTA and outperforms all vision-based 3D tracking methods in the benchmark.
  arXiv  Detail & Related papers  (2021-07-11T23:56:53Z)
• Deep traffic light detection by overlaying synthetic context on arbitrary natural images [49.592798832978296]
  We propose a method to generate artificial traffic-related training data for deep traffic light detectors. This data is generated using basic non-realistic computer graphics to blend fake traffic scenes on top of arbitrary image backgrounds. It also tackles the intrinsic data imbalance problem in traffic light datasets, caused mainly by the low amount of samples of the yellow state.
  arXiv  Detail & Related papers  (2020-11-07T19:57:22Z)
• CurbScan: Curb Detection and Tracking Using Multi-Sensor Fusion [0.8722958995761769]
  Curb detection and tracking are useful in vehicle localization and path planning. We propose an approach to detect and track curbs by fusing together data from multiple sensors. Our algorithm maintains over 90% accuracy within 4.5-22 meters and 0-14 meters for the KITTI dataset and our dataset respectively.
  arXiv  Detail & Related papers  (2020-10-09T22:48:20Z)
• Tracking Road Users using Constraint Programming [79.32806233778511]
  We present a constraint programming (CP) approach for the data association phase found in the tracking-by-detection paradigm of the multiple object tracking (MOT) problem. Our proposed method was tested on a motorized vehicles tracking dataset and produces results that outperform the top methods of the UA-DETRAC benchmark.
  arXiv  Detail & Related papers  (2020-03-10T00:04:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.