End-to-End Interactive Prediction and Planning with Optical Flow Distillation for Autonomous Driving

• URL: http://arxiv.org/abs/2104.08862v1
• Date: Sun, 18 Apr 2021 14:05:18 GMT
• Title: End-to-End Interactive Prediction and Planning with Optical Flow Distillation for Autonomous Driving
• Authors: Hengli Wang, Peide Cai, Rui Fan, Yuxiang Sun, Ming Liu
• Abstract summary: We propose an end-to-end interactive neural motion planner (INMP) for autonomous driving in this paper. Our INMP first generates a feature map in bird's-eye-view space, which is then processed to detect other agents and perform interactive prediction and planning jointly. Also, we adopt an optical flow distillation paradigm, which can effectively improve the network performance while still maintaining its real-time inference speed.
• Score: 16.340715765227475
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the recent advancement of deep learning technology, data-driven approaches for autonomous car prediction and planning have achieved extraordinary performance. Nevertheless, most of these approaches follow a non-interactive prediction and planning paradigm, hypothesizing that a vehicle's behaviors do not affect others. The approaches based on such a non-interactive philosophy typically perform acceptably in sparse traffic scenarios but can easily fail in dense traffic scenarios. Therefore, we propose an end-to-end interactive neural motion planner (INMP) for autonomous driving in this paper. Given a set of past surrounding-view images and a high definition map, our INMP first generates a feature map in bird's-eye-view space, which is then processed to detect other agents and perform interactive prediction and planning jointly. Also, we adopt an optical flow distillation paradigm, which can effectively improve the network performance while still maintaining its real-time inference speed. Extensive experiments on the nuScenes dataset and in the closed-loop Carla simulation environment demonstrate the effectiveness and efficiency of our INMP for the detection, prediction, and planning tasks. Our project page is at sites.google.com/view/inmp-ofd.

Related papers

• AMP: Autoregressive Motion Prediction Revisited with Next Token Prediction for Autonomous Driving [59.94343412438211]
  We introduce the GPT style next token motion prediction into motion prediction. Different from language data which is composed of homogeneous units -words, the elements in the driving scene could have complex spatial-temporal and semantic relations. We propose to adopt three factorized attention modules with different neighbors for information aggregation and different position encoding styles to capture their relations.
  arXiv  Detail & Related papers  (2024-03-20T06:22:37Z)
• PPAD: Iterative Interactions of Prediction and Planning for End-to-end Autonomous Driving [57.89801036693292]
  PPAD (Iterative Interaction of Prediction and Planning Autonomous Driving) considers the timestep-wise interaction to better integrate prediction and planning. We design ego-to-agent, ego-to-map, and ego-to-BEV interaction mechanisms with hierarchical dynamic key objects attention to better model the interactions.
  arXiv  Detail & Related papers  (2023-11-14T11:53:24Z)
• Implicit Occupancy Flow Fields for Perception and Prediction in Self-Driving [68.95178518732965]
  A self-driving vehicle (SDV) must be able to perceive its surroundings and predict the future behavior of other traffic participants. Existing works either perform object detection followed by trajectory of the detected objects, or predict dense occupancy and flow grids for the whole scene. This motivates our unified approach to perception and future prediction that implicitly represents occupancy and flow over time with a single neural network.
  arXiv  Detail & Related papers  (2023-08-02T23:39:24Z)
• Learning Interpretable End-to-End Vision-Based Motion Planning for Autonomous Driving with Optical Flow Distillation [11.638798976654327]
  IVMP is an interpretable end-to-end vision-based motion planning approach for autonomous driving. We develop an optical flow distillation paradigm, which can effectively enhance the network while still maintaining its real-time performance. Our IVMP significantly outperforms the state-of-the-art approaches in imitating human drivers with a much higher success rate.
  arXiv  Detail & Related papers  (2021-04-18T13:51:25Z)
• Deep Structured Reactive Planning [94.92994828905984]
  We propose a novel data-driven, reactive planning objective for self-driving vehicles. We show that our model outperforms a non-reactive variant in successfully completing highly complex maneuvers.
  arXiv  Detail & Related papers  (2021-01-18T01:43:36Z)
• The Importance of Prior Knowledge in Precise Multimodal Prediction [71.74884391209955]
  Roads have well defined geometries, topologies, and traffic rules. In this paper we propose to incorporate structured priors as a loss function. We demonstrate the effectiveness of our approach on real-world self-driving datasets.
  arXiv  Detail & Related papers  (2020-06-04T03:56:11Z)
• PiP: Planning-informed Trajectory Prediction for Autonomous Driving [69.41885900996589]
  We propose planning-informed trajectory prediction (PiP) to tackle the prediction problem in the multi-agent setting. By informing the prediction process with the planning of ego vehicle, our method achieves the state-of-the-art performance of multi-agent forecasting on highway datasets.
  arXiv  Detail & Related papers  (2020-03-25T16:09:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.