VideoGAN-based Trajectory Proposal for Automated Vehicles

• URL: http://arxiv.org/abs/2506.16209v1
• Date: Thu, 19 Jun 2025 10:57:44 GMT
• Title: VideoGAN-based Trajectory Proposal for Automated Vehicles
• Authors: Annajoyce Mariani, Kira Maag, Hanno Gottschalk,
• Abstract summary: We investigate whether a generative network (GAN) trained on videos of bird's-eye view (BEV) traffic scenarios can generate statistically accurate trajectories.<n>To this end, we propose a pipeline that uses low-resolution BEV occupancy grid videos as training data for a video generative model.<n>We obtain our best results within 100 GPU hours of training, with inference times under 20,ms.
• Score: 1.693200946453174
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Being able to generate realistic trajectory options is at the core of increasing the degree of automation of road vehicles. While model-driven, rule-based, and classical learning-based methods are widely used to tackle these tasks at present, they can struggle to effectively capture the complex, multimodal distributions of future trajectories. In this paper we investigate whether a generative adversarial network (GAN) trained on videos of bird's-eye view (BEV) traffic scenarios can generate statistically accurate trajectories that correctly capture spatial relationships between the agents. To this end, we propose a pipeline that uses low-resolution BEV occupancy grid videos as training data for a video generative model. From the generated videos of traffic scenarios we extract abstract trajectory data using single-frame object detection and frame-to-frame object matching. We particularly choose a GAN architecture for the fast training and inference times with respect to diffusion models. We obtain our best results within 100 GPU hours of training, with inference times under 20\,ms. We demonstrate the physical realism of the proposed trajectories in terms of distribution alignment of spatial and dynamic parameters with respect to the ground truth videos from the Waymo Open Motion Dataset.

Related papers

• Gradient-based Trajectory Optimization with Parallelized Differentiable Traffic Simulation [24.95575815501035]
  We present a parallelized differentiable traffic simulator based on the Intelligent Driver Model (IDM)<n>Our vehicle simulator efficiently models vehicle motion, generating trajectories that can be supervised to fit real-world data.<n>We show that we can use the simulator to filter noise in the input trajectories (trajectory filtering), reconstruct dense trajectories from sparse ones (trajectory reconstruction), and predict future trajectories.
  arXiv  Detail & Related papers  (2024-12-21T19:53:38Z)
• VeTraSS: Vehicle Trajectory Similarity Search Through Graph Modeling and Representation Learning [9.325787573209201]
  Trajectory similarity search plays an essential role in autonomous driving. We propose VeTraSS -- an end-to-end pipeline for Vehicle Trajectory Similarity Search.
  arXiv  Detail & Related papers  (2024-04-11T06:19:55Z)
• Pre-training on Synthetic Driving Data for Trajectory Prediction [61.520225216107306]
  We propose a pipeline-level solution to mitigate the issue of data scarcity in trajectory forecasting. We adopt HD map augmentation and trajectory synthesis for generating driving data, and then we learn representations by pre-training on them. We conduct extensive experiments to demonstrate the effectiveness of our data expansion and pre-training strategies.
  arXiv  Detail & Related papers  (2023-09-18T19:49:22Z)
• TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinement [64.11385310305612]
  We present a novel model for Tracking Any Point (TAP) that effectively tracks any queried point on any physical surface throughout a video sequence. Our approach employs two stages: (1) a matching stage, which independently locates a suitable candidate point match for the query point on every other frame, and (2) a refinement stage, which updates both the trajectory and query features based on local correlations. The resulting model surpasses all baseline methods by a significant margin on the TAP-Vid benchmark, as demonstrated by an approximate 20% absolute average Jaccard (AJ) improvement on DAVIS.
  arXiv  Detail & Related papers  (2023-06-14T17:07:51Z)
• Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusion [83.88829943619656]
  We introduce a method for generating realistic pedestrian trajectories and full-body animations that can be controlled to meet user-defined goals. Our guided diffusion model allows users to constrain trajectories through target waypoints, speed, and specified social groups. We propose utilizing the value function learned during RL training of the animation controller to guide diffusion to produce trajectories better suited for particular scenarios.
  arXiv  Detail & Related papers  (2023-04-04T15:46:42Z)
• TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Prediction [149.5716746789134]
  We show data-driven traffic simulation can be formulated as a world model. We present TrafficBots, a multi-agent policy built upon motion prediction and end-to-end driving. Experiments on the open motion dataset show TrafficBots can simulate realistic multi-agent behaviors.
  arXiv  Detail & Related papers  (2023-03-07T18:28:41Z)
• Street-View Image Generation from a Bird's-Eye View Layout [95.36869800896335]
  Bird's-Eye View (BEV) Perception has received increasing attention in recent years. Data-driven simulation for autonomous driving has been a focal point of recent research. We propose BEVGen, a conditional generative model that synthesizes realistic and spatially consistent surrounding images.
  arXiv  Detail & Related papers  (2023-01-11T18:39:34Z)
• Generating Synthetic Training Data for Deep Learning-Based UAV Trajectory Prediction [11.241614693184323]
  We present an approach for generating synthetic trajectory data of unmanned-aerial-vehicles (UAVs) in image space. We show that an RNN-based prediction model solely trained on the generated data can outperform classic reference models on a real-world UAV tracking dataset.
  arXiv  Detail & Related papers  (2021-07-01T13:08:31Z)
• A Deep Learning Framework for Generation and Analysis of Driving Scenario Trajectories [2.908482270923597]
  We propose a unified deep learning framework for the generation and analysis of driving scenario trajectories. We experimentally investigate the performance of the proposed framework on real-world scenario trajectories obtained from in-field data collection.
  arXiv  Detail & Related papers  (2020-07-28T23:33:05Z)
• AutoTrajectory: Label-free Trajectory Extraction and Prediction from Videos using Dynamic Points [92.91569287889203]
  We present a novel, label-free algorithm, AutoTrajectory, for trajectory extraction and prediction. To better capture the moving objects in videos, we introduce dynamic points. We aggregate dynamic points to instance points, which stand for moving objects such as pedestrians in videos.
  arXiv  Detail & Related papers  (2020-07-11T08:43:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.