Unveiling Transferability in Trajectory Prediction via Latent Scene Embeddings

• URL: http://arxiv.org/abs/2606.30777v1
• Date: Mon, 29 Jun 2026 18:07:34 GMT
• Title: Unveiling Transferability in Trajectory Prediction via Latent Scene Embeddings
• Abstract summary: Models trained on one dataset often fail to generalize beyond their training domain.<n>A framework that learns latent representations of datasets and quantifies their similarity using distributional metrics is presented.
• Score: 3.9898522485253256
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The growing availability of trajectory datasets has fueled major advances in data-driven motion prediction. Yet, models trained on one dataset often fail to generalize beyond their training domain as a result of differences in scene layouts, agent behaviors, and sensing conditions. A framework that learns latent representations of datasets and quantifies their similarity using distributional metrics is presented. This large-scale study covers 24 major datasets, including the most widely used motion-prediction benchmarks, and shows that the resulting transferability scores strongly correlate with cross-dataset model performance. The results provide practical guidance for dataset selection, pretraining, and large-scale foundation models for motion prediction, paving the way toward more generalizable and robust predictive systems.

Related papers

• Prior Distribution and Model Confidence [0.0]
  We propose a framework to understand the confidence of model predictions on unseen data without the need for retraining.<n>Our approach filters out low-confidence predictions based on their distance from the training distribution in the embedding space.<n>The proposed method is model-agnostic and generalizable, with potential applications beyond computer vision.
  arXiv  Detail & Related papers  (2025-09-05T20:17:26Z)
• Adapting to Length Shift: FlexiLength Network for Trajectory Prediction [53.637837706712794]
  Trajectory prediction plays an important role in various applications, including autonomous driving, robotics, and scene understanding. Existing approaches mainly focus on developing compact neural networks to increase prediction precision on public datasets, typically employing a standardized input duration. We introduce a general and effective framework, the FlexiLength Network (FLN), to enhance the robustness of existing trajectory prediction against varying observation periods.
  arXiv  Detail & Related papers  (2024-03-31T17:18:57Z)
• Knowledge-aware Graph Transformer for Pedestrian Trajectory Prediction [15.454206825258169]
  Predicting pedestrian motion trajectories is crucial for path planning and motion control of autonomous vehicles. Recent deep learning-based prediction approaches mainly utilize information like trajectory history and interactions between pedestrians. This paper proposes a graph transformer structure to improve prediction performance.
  arXiv  Detail & Related papers  (2024-01-10T01:50:29Z)
• Improving Transferability for Cross-domain Trajectory Prediction via Neural Stochastic Differential Equation [41.09061877498741]
  discrepancies exist among datasets due to external factors and data acquisition strategies. The proficient performance of models trained on large-scale datasets has limited transferability on other small-size datasets. We propose a method based on continuous and utilization of Neural Differential Equations (NSDE) for alleviating discrepancies. The effectiveness of our method is validated against state-of-the-art trajectory prediction models on the popular benchmark datasets: nuScenes, Argoverse, Lyft, INTERACTION, and Open Motion dataset.
  arXiv  Detail & Related papers  (2023-12-26T06:50:29Z)
• JRDB-Traj: A Dataset and Benchmark for Trajectory Forecasting in Crowds [79.00975648564483]
  Trajectory forecasting models, employed in fields such as robotics, autonomous vehicles, and navigation, face challenges in real-world scenarios. This dataset provides comprehensive data, including the locations of all agents, scene images, and point clouds, all from the robot's perspective. The objective is to predict the future positions of agents relative to the robot using raw sensory input data.
  arXiv  Detail & Related papers  (2023-11-05T18:59:31Z)
• Pre-training on Synthetic Driving Data for Trajectory Prediction [61.520225216107306]
  We propose a pipeline-level solution to mitigate the issue of data scarcity in trajectory forecasting. We adopt HD map augmentation and trajectory synthesis for generating driving data, and then we learn representations by pre-training on them. We conduct extensive experiments to demonstrate the effectiveness of our data expansion and pre-training strategies.
  arXiv  Detail & Related papers  (2023-09-18T19:49:22Z)
• In-Domain Self-Supervised Learning Improves Remote Sensing Image Scene Classification [5.323049242720532]
  Self-supervised learning has emerged as a promising approach for remote sensing image classification. We present a study of different self-supervised pre-training strategies and evaluate their effect across 14 downstream datasets.
  arXiv  Detail & Related papers  (2023-07-04T10:57:52Z)
• On the Trade-off of Intra-/Inter-class Diversity for Supervised Pre-training [72.8087629914444]
  We study the impact of the trade-off between the intra-class diversity (the number of samples per class) and the inter-class diversity (the number of classes) of a supervised pre-training dataset. With the size of the pre-training dataset fixed, the best downstream performance comes with a balance on the intra-/inter-class diversity.
  arXiv  Detail & Related papers  (2023-05-20T16:23:50Z)
• Beyond Transfer Learning: Co-finetuning for Action Localisation [64.07196901012153]
  We propose co-finetuning -- simultaneously training a single model on multiple upstream'' and downstream'' tasks. We demonstrate that co-finetuning outperforms traditional transfer learning when using the same total amount of data. We also show how we can easily extend our approach to multiple upstream'' datasets to further improve performance.
  arXiv  Detail & Related papers  (2022-07-08T10:25:47Z)
• MEAT: Maneuver Extraction from Agent Trajectories [9.919575841909962]
  We propose an automated methodology that allows to extract maneuvers from agent trajectories in large-scale datasets. Although it is possible to use the resulting maneuvers for training classification networks, we exemplary use them for extensive trajectory dataset analysis.
  arXiv  Detail & Related papers  (2022-06-10T14:56:32Z)
• How Well Do Sparse Imagenet Models Transfer? [75.98123173154605]
  Transfer learning is a classic paradigm by which models pretrained on large "upstream" datasets are adapted to yield good results on "downstream" datasets. In this work, we perform an in-depth investigation of this phenomenon in the context of convolutional neural networks (CNNs) trained on the ImageNet dataset. We show that sparse models can match or even outperform the transfer performance of dense models, even at high sparsities.
  arXiv  Detail & Related papers  (2021-11-26T11:58:51Z)
• Improved Fine-tuning by Leveraging Pre-training Data: Theory and Practice [52.11183787786718]
  Fine-tuning a pre-trained model on the target data is widely used in many deep learning applications. Recent studies have empirically shown that training from scratch has the final performance that is no worse than this pre-training strategy. We propose a novel selection strategy to select a subset from pre-training data to help improve the generalization on the target task.
  arXiv  Detail & Related papers  (2021-11-24T06:18:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.