roadscene2vec: A Tool for Extracting and Embedding Road Scene-Graphs

• URL: http://arxiv.org/abs/2109.01183v1
• Date: Thu, 2 Sep 2021 19:21:18 GMT
• Title: roadscene2vec: A Tool for Extracting and Embedding Road Scene-Graphs
• Authors: Arnav Vaibhav Malawade, Shih-Yuan Yu, Brandon Hsu, Harsimrat Kaeley, Anurag Karra, Mohammad Abdullah Al Faruque
• Abstract summary: roadscene2vec is an open-source tool for extracting and embedding road scene-graphs. The goal of roadscene2vec is to enable research into applications and capabilities of road scene-graphs.
• Score: 3.9482018271405073
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, road scene-graph representations used in conjunction with graph learning techniques have been shown to outperform state-of-the-art deep learning techniques in tasks including action classification, risk assessment, and collision prediction. To enable the exploration of applications of road scene-graph representations, we introduce roadscene2vec: an open-source tool for extracting and embedding road scene-graphs. The goal of roadscene2vec is to enable research into the applications and capabilities of road scene-graphs by providing tools for generating scene-graphs, graph learning models to generate spatio-temporal scene-graph embeddings, and tools for visualizing and analyzing scene-graph-based methodologies. The capabilities of roadscene2vec include (i) customized scene-graph generation from either video clips or data from the CARLA simulator, (ii) multiple configurable spatio-temporal graph embedding models and baseline CNN-based models, (iii) built-in functionality for using graph and sequence embeddings for risk assessment and collision prediction applications, (iv) tools for evaluating transfer learning, and (v) utilities for visualizing scene-graphs and analyzing the explainability of graph learning models. We demonstrate the utility of roadscene2vec for these use cases with experimental results and qualitative evaluations for both graph learning models and CNN-based models. roadscene2vec is available at https://github.com/AICPS/roadscene2vec.

Related papers

• nuScenes Knowledge Graph -- A comprehensive semantic representation of traffic scenes for trajectory prediction [6.23221362105447]
  Trajectory prediction in traffic scenes involves accurately forecasting the behaviour of surrounding vehicles. It is crucial to consider contextual information, including the driving path of vehicles, road topology, lane dividers, and traffic rules. This paper presents an approach that utilizes knowledge graphs to model the diverse entities and their semantic connections within traffic scenes.
  arXiv  Detail & Related papers  (2023-12-15T10:40:34Z)
• Open-Vocabulary Object Detection via Scene Graph Discovery [53.27673119360868]
  Open-vocabulary (OV) object detection has attracted increasing research attention. We propose a novel Scene-Graph-Based Discovery Network (SGDN) that exploits scene graph cues for OV detection.
  arXiv  Detail & Related papers  (2023-07-07T00:46:19Z)
• CommonScenes: Generating Commonsense 3D Indoor Scenes with Scene Graph Diffusion [83.30168660888913]
  We present CommonScenes, a fully generative model that converts scene graphs into corresponding controllable 3D scenes. Our pipeline consists of two branches, one predicting the overall scene layout via a variational auto-encoder and the other generating compatible shapes. The generated scenes can be manipulated by editing the input scene graph and sampling the noise in the diffusion model.
  arXiv  Detail & Related papers  (2023-05-25T17:39:13Z)
• Iterative Scene Graph Generation with Generative Transformers [6.243995448840211]
  Scene graphs provide a rich, structured representation of a scene by encoding the entities (objects) and their spatial relationships in a graphical format. Current approaches take a generation-by-classification approach where the scene graph is generated through labeling of all possible edges between objects in a scene. This work introduces a generative transformer-based approach to generating scene graphs beyond link prediction.
  arXiv  Detail & Related papers  (2022-11-30T00:05:44Z)
• Learning to Generate Scene Graph from Natural Language Supervision [52.18175340725455]
  We propose one of the first methods that learn from image-sentence pairs to extract a graphical representation of localized objects and their relationships within an image, known as scene graph. We leverage an off-the-shelf object detector to identify and localize object instances, match labels of detected regions to concepts parsed from captions, and thus create "pseudo" labels for learning scene graph.
  arXiv  Detail & Related papers  (2021-09-06T03:38:52Z)
• Graph-to-3D: End-to-End Generation and Manipulation of 3D Scenes Using Scene Graphs [85.54212143154986]
  Controllable scene synthesis consists of generating 3D information that satisfy underlying specifications. Scene graphs are representations of a scene composed of objects (nodes) and inter-object relationships (edges) We propose the first work that directly generates shapes from a scene graph in an end-to-end manner.
  arXiv  Detail & Related papers  (2021-08-19T17:59:07Z)
• Lane Graph Estimation for Scene Understanding in Urban Driving [34.82775302794312]
  We propose a novel approach for lane geometry estimation from bird's-eye-view images. We train a graph estimation model on multimodal bird's-eye-view data processed from the popular NuScenes dataset. Our model shows promising performance for most evaluated urban scenes and can serve as a step towards automated generation of HD lane annotations for autonomous driving.
  arXiv  Detail & Related papers  (2021-05-01T08:38:18Z)
• SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequences [76.28527350263012]
  We propose a method to incrementally build up semantic scene graphs from a 3D environment given a sequence of RGB-D frames. We aggregate PointNet features from primitive scene components by means of a graph neural network. Our approach outperforms 3D scene graph prediction methods by a large margin and its accuracy is on par with other 3D semantic and panoptic segmentation methods while running at 35 Hz.
  arXiv  Detail & Related papers  (2021-03-27T13:00:36Z)
• Road Scene Graph: A Semantic Graph-Based Scene Representation Dataset for Intelligent Vehicles [72.04891523115535]
  We propose road scene graph,a special scene-graph for intelligent vehicles. It provides not only object proposals but also their pair-wise relationships. By organizing them in a topological graph, these data are explainable, fully-connected, and could be easily processed by GCNs.
  arXiv  Detail & Related papers  (2020-11-27T07:33:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.