Trajectory Prediction with Linguistic Representations
- URL: http://arxiv.org/abs/2110.09741v1
- Date: Tue, 19 Oct 2021 05:22:38 GMT
- Title: Trajectory Prediction with Linguistic Representations
- Authors: Yen-Ling Kuo, Xin Huang, Andrei Barbu, Stephen G. McGill, Boris Katz,
John J. Leonard, Guy Rosman
- Abstract summary: We present a novel trajectory prediction model that uses linguistic intermediate representations to forecast trajectories.
The model learns the meaning of each of the words without direct per-word supervision.
It generates a linguistic description of trajectories which captures maneuvers and interactions over an extended time interval.
- Score: 27.71805777845141
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Language allows humans to build mental models that interpret what is
happening around them resulting in more accurate long-term predictions. We
present a novel trajectory prediction model that uses linguistic intermediate
representations to forecast trajectories, and is trained using trajectory
samples with partially annotated captions. The model learns the meaning of each
of the words without direct per-word supervision. At inference time, it
generates a linguistic description of trajectories which captures maneuvers and
interactions over an extended time interval. This generated description is used
to refine predictions of the trajectories of multiple agents. We train and
validate our model on the Argoverse dataset, and demonstrate improved accuracy
results in trajectory prediction. In addition, our model is more interpretable:
it presents part of its reasoning in plain language as captions, which can aid
model development and can aid in building confidence in the model before
deploying it.
Related papers
- Traj-Explainer: An Explainable and Robust Multi-modal Trajectory Prediction Approach [12.60529039445456]
Navigating complex traffic environments has been significantly enhanced by advancements in intelligent technologies, enabling accurate environment perception and trajectory prediction for automated vehicles.
Existing research often neglects the consideration of the joint reasoning of scenario agents and lacks interpretability in trajectory prediction models.
An explainability-oriented trajectory prediction model is designed in this work, named Explainable Diffusion Conditional based Multimodal Trajectory Prediction Traj-Explainer.
arXiv Detail & Related papers (2024-10-22T08:17:33Z) - Towards Generalizable and Interpretable Motion Prediction: A Deep
Variational Bayes Approach [54.429396802848224]
This paper proposes an interpretable generative model for motion prediction with robust generalizability to out-of-distribution cases.
For interpretability, the model achieves the target-driven motion prediction by estimating the spatial distribution of long-term destinations.
Experiments on motion prediction datasets validate that the fitted model can be interpretable and generalizable.
arXiv Detail & Related papers (2024-03-10T04:16:04Z) - Large language models implicitly learn to straighten neural sentence
trajectories to construct a predictive representation of natural language [2.1756081703276]
We test a hypothesis about predictive representations of autoregressive transformers.
Key insight is that straighter trajectories should facilitate prediction via linear extrapolation.
We quantify straightness using a 1-dimensional curvature metric.
arXiv Detail & Related papers (2023-11-05T22:16:21Z) - Improving Trajectory Prediction in Dynamic Multi-Agent Environment by
Dropping Waypoints [9.385936248154987]
Motion prediction systems must learn spatial and temporal information from the past to forecast the future trajectories of the agent.
We propose Temporal Waypoint Dropping (TWD) that explicitly incorporates temporal dependencies during the training of a trajectory prediction model.
We evaluate our proposed approach on three datasets: NBA Sports VU, ETH-UCY, and TrajNet++.
arXiv Detail & Related papers (2023-09-29T15:48:35Z) - How to Estimate Model Transferability of Pre-Trained Speech Models? [84.11085139766108]
"Score-based assessment" framework for estimating transferability of pre-trained speech models.
We leverage upon two representation theories, Bayesian likelihood estimation and optimal transport, to generate rank scores for the PSM candidates.
Our framework efficiently computes transferability scores without actual fine-tuning of candidate models or layers.
arXiv Detail & Related papers (2023-06-01T04:52:26Z) - Explaining How Transformers Use Context to Build Predictions [0.1749935196721634]
Language Generation Models produce words based on the previous context.
It is still unclear how prior words affect the model's decision throughout the layers.
We leverage recent advances in explainability of the Transformer and present a procedure to analyze models for language generation.
arXiv Detail & Related papers (2023-05-21T18:29:10Z) - Few-shot Subgoal Planning with Language Models [58.11102061150875]
We show that language priors encoded in pre-trained language models allow us to infer fine-grained subgoal sequences.
In contrast to recent methods which make strong assumptions about subgoal supervision, our experiments show that language models can infer detailed subgoal sequences without any fine-tuning.
arXiv Detail & Related papers (2022-05-28T01:03:30Z) - Pathologies of Pre-trained Language Models in Few-shot Fine-tuning [50.3686606679048]
We show that pre-trained language models with few examples show strong prediction bias across labels.
Although few-shot fine-tuning can mitigate the prediction bias, our analysis shows models gain performance improvement by capturing non-task-related features.
These observations alert that pursuing model performance with fewer examples may incur pathological prediction behavior.
arXiv Detail & Related papers (2022-04-17T15:55:18Z) - A Generative Language Model for Few-shot Aspect-Based Sentiment Analysis [90.24921443175514]
We focus on aspect-based sentiment analysis, which involves extracting aspect term, category, and predicting their corresponding polarities.
We propose to reformulate the extraction and prediction tasks into the sequence generation task, using a generative language model with unidirectional attention.
Our approach outperforms the previous state-of-the-art (based on BERT) on average performance by a large margins in few-shot and full-shot settings.
arXiv Detail & Related papers (2022-04-11T18:31:53Z) - You Mostly Walk Alone: Analyzing Feature Attribution in Trajectory
Prediction [52.442129609979794]
Recent deep learning approaches for trajectory prediction show promising performance.
It remains unclear which features such black-box models actually learn to use for making predictions.
This paper proposes a procedure that quantifies the contributions of different cues to model performance.
arXiv Detail & Related papers (2021-10-11T14:24:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.