Forecasting from LiDAR via Future Object Detection

• URL: http://arxiv.org/abs/2203.16297v2
• Date: Thu, 31 Mar 2022 14:17:09 GMT
• Title: Forecasting from LiDAR via Future Object Detection
• Authors: Neehar Peri, Jonathon Luiten, Mengtian Li, Aljo\v{s}a O\v{s}ep, Laura Leal-Taix\'e, Deva Ramanan
• Abstract summary: We propose an end-to-end approach for detection and motion forecasting based on raw sensor measurement. By linking future and current locations in a many-to-one manner, our approach is able to reason about multiple futures.
• Score: 47.11167997187244
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Object detection and forecasting are fundamental components of embodied perception. These two problems, however, are largely studied in isolation by the community. In this paper, we propose an end-to-end approach for detection and motion forecasting based on raw sensor measurement as opposed to ground truth tracks. Instead of predicting the current frame locations and forecasting forward in time, we directly predict future object locations and backcast to determine where each trajectory began. Our approach not only improves overall accuracy compared to other modular or end-to-end baselines, it also prompts us to rethink the role of explicit tracking for embodied perception. Additionally, by linking future and current locations in a many-to-one manner, our approach is able to reason about multiple futures, a capability that was previously considered difficult for end-to-end approaches. We conduct extensive experiments on the popular nuScenes dataset and demonstrate the empirical effectiveness of our approach. In addition, we investigate the appropriateness of reusing standard forecasting metrics for an end-to-end setup, and find a number of limitations which allow us to build simple baselines to game these metrics. We address this issue with a novel set of joint forecasting and detection metrics that extend the commonly used AP metrics from the detection community to measuring forecasting accuracy. Our code is available at https://github.com/neeharperi/FutureDet

Related papers

• Valeo4Cast: A Modular Approach to End-to-End Forecasting [93.86257326005726]
  Our solution ranks first in the Argoverse 2 End-to-end Forecasting Challenge, with 63.82 mAPf. We depart from the current trend of tackling this task via end-to-end training from perception to forecasting, and instead use a modular approach. We surpass forecasting results by +17.1 points over last year's winner and by +13.3 points over this year's runner-up.
  arXiv  Detail & Related papers  (2024-06-12T11:50:51Z)
• JRDB-Traj: A Dataset and Benchmark for Trajectory Forecasting in Crowds [79.00975648564483]
  Trajectory forecasting models, employed in fields such as robotics, autonomous vehicles, and navigation, face challenges in real-world scenarios. This dataset provides comprehensive data, including the locations of all agents, scene images, and point clouds, all from the robot's perspective. The objective is to predict the future positions of agents relative to the robot using raw sensory input data.
  arXiv  Detail & Related papers  (2023-11-05T18:59:31Z)
• Towards Motion Forecasting with Real-World Perception Inputs: Are End-to-End Approaches Competitive? [93.10694819127608]
  We propose a unified evaluation pipeline for forecasting methods with real-world perception inputs. Our in-depth study uncovers a substantial performance gap when transitioning from curated to perception-based data.
  arXiv  Detail & Related papers  (2023-06-15T17:03:14Z)
• LMD: Light-weight Prediction Quality Estimation for Object Detection in Lidar Point Clouds [3.927702899922668]
  Object detection on Lidar point cloud data is a promising technology for autonomous driving and robotics. Uncertainty estimation is a crucial component for down-stream tasks and deep neural networks remain error-prone even for predictions with high confidence. We propose LidarMetaDetect, a light-weight post-processing scheme for prediction quality estimation. Our experiments show a significant increase of statistical reliability in separating true from false predictions.
  arXiv  Detail & Related papers  (2023-06-13T15:13:29Z)
• Pedestrian Trajectory Forecasting Using Deep Ensembles Under Sensing Uncertainty [125.41260574344933]
  We consider an encoder-decoder based deep ensemble network for capturing both perception and predictive uncertainty simultaneously. Overall, deep ensembles provided more robust predictions and the consideration of upstream uncertainty further increased the estimation accuracy for the model.
  arXiv  Detail & Related papers  (2023-05-26T04:27:48Z)
• Trajectory Forecasting from Detection with Uncertainty-Aware Motion Encoding [121.66374635092097]
  Trajectories obtained from object detection and tracking are inevitably noisy. We propose a trajectory predictor directly based on detection results without relying on explicitly formed trajectories.
  arXiv  Detail & Related papers  (2022-02-03T09:09:56Z)
• MTP: Multi-Hypothesis Tracking and Prediction for Reduced Error Propagation [39.41917241231786]
  This paper addresses the problem of cascading errors by focusing on the coupling between the tracking and prediction modules. By using state-of-the-art tracking and prediction tools, we conduct a comprehensive experimental evaluation of how severely errors stemming from tracking can impact prediction performance. We show that this framework improves overall prediction performance over the standard single-hypothesis tracking-prediction pipeline by up to 34.2% on the nuScenes dataset.
  arXiv  Detail & Related papers  (2021-10-18T17:30:59Z)
• Online Co-movement Pattern Prediction in Mobility Data [1.5790464310310084]
  We provide an accurate solution to the problem of Online Prediction of Co-movement patterns. In order to calculate the accuracy of our solution, we propose a co-movement pattern similarity measure. The accuracy of our solution is demonstrated experimentally over a real dataset from the maritime domain.
  arXiv  Detail & Related papers  (2021-02-17T17:01:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.