MANTRA: Memory Augmented Networks for Multiple Trajectory Prediction
- URL: http://arxiv.org/abs/2006.03340v2
- Date: Thu, 3 Jun 2021 22:52:06 GMT
- Title: MANTRA: Memory Augmented Networks for Multiple Trajectory Prediction
- Authors: Francesco Marchetti, Federico Becattini, Lorenzo Seidenari, Alberto
Del Bimbo
- Abstract summary: We address the problem of multimodal trajectory prediction exploiting a Memory Augmented Neural Network.
Our method learns past and future trajectory embeddings using recurrent neural networks and exploits an associative external memory to store and retrieve such embeddings.
Trajectory prediction is then performed by decoding in-memory future encodings conditioned with the observed past.
- Score: 26.151761714896118
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Autonomous vehicles are expected to drive in complex scenarios with several
independent non cooperating agents. Path planning for safely navigating in such
environments can not just rely on perceiving present location and motion of
other agents. It requires instead to predict such variables in a far enough
future. In this paper we address the problem of multimodal trajectory
prediction exploiting a Memory Augmented Neural Network. Our method learns past
and future trajectory embeddings using recurrent neural networks and exploits
an associative external memory to store and retrieve such embeddings.
Trajectory prediction is then performed by decoding in-memory future encodings
conditioned with the observed past. We incorporate scene knowledge in the
decoding state by learning a CNN on top of semantic scene maps. Memory growth
is limited by learning a writing controller based on the predictive capability
of existing embeddings. We show that our method is able to natively perform
multi-modal trajectory prediction obtaining state-of-the art results on three
datasets. Moreover, thanks to the non-parametric nature of the memory module,
we show how once trained our system can continuously improve by ingesting novel
patterns.
Related papers
- Towards Consistent and Explainable Motion Prediction using Heterogeneous Graph Attention [0.17476232824732776]
This paper introduces a new refinement module designed to project the predicted trajectories back onto the actual map.
We also propose a novel scene encoder that handles all relations between agents and their environment in a single unified graph attention network.
arXiv Detail & Related papers (2024-05-16T14:31:15Z) - Memory-and-Anticipation Transformer for Online Action Understanding [52.24561192781971]
We propose a novel memory-anticipation-based paradigm to model an entire temporal structure, including the past, present, and future.
We present Memory-and-Anticipation Transformer (MAT), a memory-anticipation-based approach, to address the online action detection and anticipation tasks.
arXiv Detail & Related papers (2023-08-15T17:34:54Z) - Hidden State Approximation in Recurrent Neural Networks Using Continuous
Particle Filtering [0.0]
Using historical data to predict future events has many applications in the real world, such as stock price prediction; the robot localization.
In this paper, we use the particles to approximate the distribution of the latent state and show how it can extend into a more complex form.
With the proposed continuous differentiable scheme, our model is capable of adaptively extracting valuable information and updating the latent state according to the Bayes rule.
arXiv Detail & Related papers (2022-12-18T04:31:45Z) - Interpretable Self-Aware Neural Networks for Robust Trajectory
Prediction [50.79827516897913]
We introduce an interpretable paradigm for trajectory prediction that distributes the uncertainty among semantic concepts.
We validate our approach on real-world autonomous driving data, demonstrating superior performance over state-of-the-art baselines.
arXiv Detail & Related papers (2022-11-16T06:28:20Z) - A Framework for Multisensory Foresight for Embodied Agents [11.351546861334292]
Predicting future sensory states is crucial for learning agents such as robots, drones, and autonomous vehicles.
In this paper, we couple multiple sensory modalities with exploratory actions and propose a predictive neural network architecture to address this problem.
The framework was tested and validated with a dataset containing 4 sensory modalities (vision, haptic, audio, and tactile) on a humanoid robot performing 9 behaviors multiple times on a large set of objects.
arXiv Detail & Related papers (2021-09-15T20:20:04Z) - PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive
Learning [109.84770951839289]
We present PredRNN, a new recurrent network for learning visual dynamics from historical context.
We show that our approach obtains highly competitive results on three standard datasets.
arXiv Detail & Related papers (2021-03-17T08:28:30Z) - SLPC: a VRNN-based approach for stochastic lidar prediction and
completion in autonomous driving [63.87272273293804]
We propose a new LiDAR prediction framework that is based on generative models namely Variational Recurrent Neural Networks (VRNNs)
Our algorithm is able to address the limitations of previous video prediction frameworks when dealing with sparse data by spatially inpainting the depth maps in the upcoming frames.
We present a sparse version of VRNNs and an effective self-supervised training method that does not require any labels.
arXiv Detail & Related papers (2021-02-19T11:56:44Z) - Unsupervised Transfer Learning for Spatiotemporal Predictive Networks [90.67309545798224]
We study how to transfer knowledge from a zoo of unsupervisedly learned models towards another network.
Our motivation is that models are expected to understand complex dynamics from different sources.
Our approach yields significant improvements on three benchmarks fortemporal prediction, and benefits the target even from less relevant ones.
arXiv Detail & Related papers (2020-09-24T15:40:55Z) - SMART: Simultaneous Multi-Agent Recurrent Trajectory Prediction [72.37440317774556]
We propose advances that address two key challenges in future trajectory prediction.
multimodality in both training data and predictions and constant time inference regardless of number of agents.
arXiv Detail & Related papers (2020-07-26T08:17:10Z) - AMENet: Attentive Maps Encoder Network for Trajectory Prediction [35.22312783822563]
Trajectory prediction is critical for applications of planning safe future movements.
We propose an end-to-end generative model named Attentive Maps Network (AMENet)
AMENet encodes the agent's motion and interaction information for accurate and realistic multi-path trajectory prediction.
arXiv Detail & Related papers (2020-06-15T10:00:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.