Instance-Aware Predictive Navigation in Multi-Agent Environments
- URL: http://arxiv.org/abs/2101.05893v1
- Date: Thu, 14 Jan 2021 22:21:25 GMT
- Title: Instance-Aware Predictive Navigation in Multi-Agent Environments
- Authors: Jinkun Cao, Xin Wang, Trevor Darrell, Fisher Yu
- Abstract summary: We propose an Instance-Aware Predictive Control (IPC) approach, which forecasts interactions between agents as well as future scene structures.
We adopt a novel multi-instance event prediction module to estimate the possible interaction among agents in the ego-centric view.
We design a sequential action sampling strategy to better leverage predicted states on both scene-level and instance-level.
- Score: 93.15055834395304
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In this work, we aim to achieve efficient end-to-end learning of driving
policies in dynamic multi-agent environments. Predicting and anticipating
future events at the object level are critical for making informed driving
decisions. We propose an Instance-Aware Predictive Control (IPC) approach,
which forecasts interactions between agents as well as future scene structures.
We adopt a novel multi-instance event prediction module to estimate the
possible interaction among agents in the ego-centric view, conditioned on the
selected action sequence of the ego-vehicle. To decide the action at each step,
we seek the action sequence that can lead to safe future states based on the
prediction module outputs by repeatedly sampling likely action sequences. We
design a sequential action sampling strategy to better leverage predicted
states on both scene-level and instance-level. Our method establishes a new
state of the art in the challenging CARLA multi-agent driving simulation
environments without expert demonstration, giving better explainability and
sample efficiency.
Related papers
- PPAD: Iterative Interactions of Prediction and Planning for End-to-end Autonomous Driving [57.89801036693292]
PPAD (Iterative Interaction of Prediction and Planning Autonomous Driving) considers the timestep-wise interaction to better integrate prediction and planning.
We design ego-to-agent, ego-to-map, and ego-to-BEV interaction mechanisms with hierarchical dynamic key objects attention to better model the interactions.
arXiv Detail & Related papers (2023-11-14T11:53:24Z) - Pixel State Value Network for Combined Prediction and Planning in
Interactive Environments [9.117828575880303]
This work proposes a deep learning methodology to combine prediction and planning.
A conditional GAN with the U-Net architecture is trained to predict two high-resolution image sequences.
Results demonstrate intuitive behavior in complex situations, such as lane changes amidst conflicting objectives.
arXiv Detail & Related papers (2023-10-11T17:57:13Z) - MotionLM: Multi-Agent Motion Forecasting as Language Modeling [15.317827804763699]
We present MotionLM, a language model for multi-agent motion prediction.
Our approach bypasses post-hoc interactions where individual agent trajectory generation is conducted prior to interactive scoring.
The model's sequential factorization enables temporally causal conditional rollouts.
arXiv Detail & Related papers (2023-09-28T15:46:25Z) - Deep Interactive Motion Prediction and Planning: Playing Games with
Motion Prediction Models [162.21629604674388]
This work presents a game-theoretic Model Predictive Controller (MPC) that uses a novel interactive multi-agent neural network policy as part of its predictive model.
Fundamental to the success of our method is the design of a novel multi-agent policy network that can steer a vehicle given the state of the surrounding agents and the map information.
arXiv Detail & Related papers (2022-04-05T17:58:18Z) - LatentFormer: Multi-Agent Transformer-Based Interaction Modeling and
Trajectory Prediction [12.84508682310717]
We propose LatentFormer, a transformer-based model for predicting future vehicle trajectories.
We evaluate the proposed method on the nuScenes benchmark dataset and show that our approach achieves state-of-the-art performance and improves upon trajectory metrics by up to 40%.
arXiv Detail & Related papers (2022-03-03T17:44:58Z) - Scene Transformer: A unified multi-task model for behavior prediction
and planning [42.758178896204036]
We formulate a model for predicting the behavior of all agents jointly in real-world driving environments.
Inspired by recent language modeling approaches, we use a masking strategy as the query to our model.
We evaluate our approach on autonomous driving datasets for behavior prediction, and achieve state-of-the-art performance.
arXiv Detail & Related papers (2021-06-15T20:20:44Z) - Contingencies from Observations: Tractable Contingency Planning with
Learned Behavior Models [82.34305824719101]
Humans have a remarkable ability to make decisions by accurately reasoning about future events.
We develop a general-purpose contingency planner that is learned end-to-end using high-dimensional scene observations.
We show how this model can tractably learn contingencies from behavioral observations.
arXiv Detail & Related papers (2021-04-21T14:30:20Z) - LookOut: Diverse Multi-Future Prediction and Planning for Self-Driving [139.33800431159446]
LookOut is an approach to jointly perceive the environment and predict a diverse set of futures from sensor data.
We show that our model demonstrates significantly more diverse and sample-efficient motion forecasting in a large-scale self-driving dataset.
arXiv Detail & Related papers (2021-01-16T23:19:22Z) - Prediction by Anticipation: An Action-Conditional Prediction Method
based on Interaction Learning [23.321627835039934]
We propose prediction by anticipation, which views interaction in terms of a latent probabilistic generative process.
Under this view, consecutive data frames can be factorized into sequential samples from an action-conditional distribution.
Our proposed prediction model, variational Bayesian in nature, is trained to maximize the evidence lower bound (ELBO) of this conditional distribution.
arXiv Detail & Related papers (2020-12-25T01:39:26Z) - SMART: Simultaneous Multi-Agent Recurrent Trajectory Prediction [72.37440317774556]
We propose advances that address two key challenges in future trajectory prediction.
multimodality in both training data and predictions and constant time inference regardless of number of agents.
arXiv Detail & Related papers (2020-07-26T08:17:10Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.