nuScenes Knowledge Graph -- A comprehensive semantic representation of traffic scenes for trajectory prediction

• URL: http://arxiv.org/abs/2312.09676v1
• Date: Fri, 15 Dec 2023 10:40:34 GMT
• Title: nuScenes Knowledge Graph -- A comprehensive semantic representation of traffic scenes for trajectory prediction
• Authors: Leon Mlodzian, Zhigang Sun, Hendrik Berkemeyer, Sebastian Monka, Zixu Wang, Stefan Dietze, Lavdim Halilaj, Juergen Luettin
• Abstract summary: Trajectory prediction in traffic scenes involves accurately forecasting the behaviour of surrounding vehicles. It is crucial to consider contextual information, including the driving path of vehicles, road topology, lane dividers, and traffic rules. This paper presents an approach that utilizes knowledge graphs to model the diverse entities and their semantic connections within traffic scenes.
• Score: 6.23221362105447
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Trajectory prediction in traffic scenes involves accurately forecasting the behaviour of surrounding vehicles. To achieve this objective it is crucial to consider contextual information, including the driving path of vehicles, road topology, lane dividers, and traffic rules. Although studies demonstrated the potential of leveraging heterogeneous context for improving trajectory prediction, state-of-the-art deep learning approaches still rely on a limited subset of this information. This is mainly due to the limited availability of comprehensive representations. This paper presents an approach that utilizes knowledge graphs to model the diverse entities and their semantic connections within traffic scenes. Further, we present nuScenes Knowledge Graph (nSKG), a knowledge graph for the nuScenes dataset, that models explicitly all scene participants and road elements, as well as their semantic and spatial relationships. To facilitate the usage of the nSKG via graph neural networks for trajectory prediction, we provide the data in a format, ready-to-use by the PyG library. All artefacts can be found here: https://github.com/boschresearch/nuScenes_Knowledge_Graph

Related papers

• Towards Consistent and Explainable Motion Prediction using Heterogeneous Graph Attention [0.17476232824732776]
  This paper introduces a new refinement module designed to project the predicted trajectories back onto the actual map. We also propose a novel scene encoder that handles all relations between agents and their environment in a single unified graph attention network.
  arXiv  Detail & Related papers  (2024-05-16T14:31:15Z)
• SemanticFormer: Holistic and Semantic Traffic Scene Representation for Trajectory Prediction using Knowledge Graphs [3.733790302392792]
  Tray prediction in autonomous driving relies on accurate representation of all relevant contexts of the driving scene. We present SemanticFormer, an approach for predicting multimodal trajectories by reasoning over a traffic scene graph.
  arXiv  Detail & Related papers  (2024-04-30T09:11:04Z)
• Unifying Lane-Level Traffic Prediction from a Graph Structural Perspective: Benchmark and Baseline [21.37853568400125]
  This paper extensively analyzes and categorizes existing research in lane-level traffic prediction. It introduces a simple baseline model, GraphMLP, based on graph structure and prediction networks. We have replicated codes not publicly available in existing studies and assessed various models in terms of effectiveness, efficiency, and applicability.
  arXiv  Detail & Related papers  (2024-03-22T04:21:40Z)
• Pre-training on Synthetic Driving Data for Trajectory Prediction [61.520225216107306]
  We propose a pipeline-level solution to mitigate the issue of data scarcity in trajectory forecasting. We adopt HD map augmentation and trajectory synthesis for generating driving data, and then we learn representations by pre-training on them. We conduct extensive experiments to demonstrate the effectiveness of our data expansion and pre-training strategies.
  arXiv  Detail & Related papers  (2023-09-18T19:49:22Z)
• Learning to Predict Navigational Patterns from Partial Observations [63.04492958425066]
  This paper presents the first self-supervised learning (SSL) method for learning to infer navigational patterns in real-world environments from partial observations only. We demonstrate how to infer global navigational patterns by fitting a maximum likelihood graph to the DSLP field. Experiments show that our SSL model outperforms two SOTA supervised lane graph prediction models on the nuScenes dataset.
  arXiv  Detail & Related papers  (2023-04-26T02:08:46Z)
• Geometric Deep Learning for Autonomous Driving: Unlocking the Power of Graph Neural Networks With CommonRoad-Geometric [6.638385593789309]
  Heterogeneous graphs offer powerful data representations for traffic, given their ability to model the complex interaction effects. With the advent of graph neural networks (GNNs) as the accompanying deep learning framework, the graph structure can be efficiently leveraged for various machine learning applications. Our proposed Python framework offers an easy-to-use and fully customizable data processing pipeline to extract standardized graph datasets from traffic scenarios.
  arXiv  Detail & Related papers  (2023-02-02T17:45:02Z)
• RSG-Net: Towards Rich Sematic Relationship Prediction for Intelligent Vehicle in Complex Environments [72.04891523115535]
  We propose RSG-Net (Road Scene Graph Net): a graph convolutional network designed to predict potential semantic relationships from object proposals. The experimental results indicate that this network, trained on Road Scene Graph dataset, could efficiently predict potential semantic relationships among objects around the ego-vehicle.
  arXiv  Detail & Related papers  (2022-07-16T12:40:17Z)
• PreTraM: Self-Supervised Pre-training via Connecting Trajectory and Map [58.53373202647576]
  We propose PreTraM, a self-supervised pre-training scheme for trajectory forecasting. It consists of two parts: 1) Trajectory-Map Contrastive Learning, where we project trajectories and maps to a shared embedding space with cross-modal contrastive learning, and 2) Map Contrastive Learning, where we enhance map representation with contrastive learning on large quantities of HD-maps. On top of popular baselines such as AgentFormer and Trajectron++, PreTraM boosts their performance by 5.5% and 6.9% relatively in FDE-10 on the challenging nuScenes dataset.
  arXiv  Detail & Related papers  (2022-04-21T23:01:21Z)
• Road Scene Graph: A Semantic Graph-Based Scene Representation Dataset for Intelligent Vehicles [72.04891523115535]
  We propose road scene graph,a special scene-graph for intelligent vehicles. It provides not only object proposals but also their pair-wise relationships. By organizing them in a topological graph, these data are explainable, fully-connected, and could be easily processed by GCNs.
  arXiv  Detail & Related papers  (2020-11-27T07:33:11Z)
• Constructing Geographic and Long-term Temporal Graph for Traffic Forecasting [88.5550074808201]
  We propose Geographic and Long term Temporal Graph Convolutional Recurrent Neural Network (GLT-GCRNN) for traffic forecasting. In this work, we propose a novel framework for traffic forecasting that learns the rich interactions between roads sharing similar geographic or longterm temporal patterns.
  arXiv  Detail & Related papers  (2020-04-23T03:50:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.