PlanTRansformer: Unified Prediction and Planning with Goal-conditioned Transformer

• URL: http://arxiv.org/abs/2602.03376v1
• Date: Tue, 03 Feb 2026 10:55:05 GMT
• Title: PlanTRansformer: Unified Prediction and Planning with Goal-conditioned Transformer
• Authors: Constantin Selzer, Fabina B. Flohr,
• Abstract summary: Trajectory prediction and planning are fundamental yet disconnected components in autonomous driving.<n>Plan TRansformer (PTR) integrates goal-conditioned prediction, dynamic feasibility, interaction awareness, and lane-level topology reasoning.<n>PTR achieves 4.3%/3.5% improvement in marginal/joint mAP compared to the baseline Motion Transformer (MTR) and 15.5% planning error reduction at 5s horizon compared to GameFormer.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Trajectory prediction and planning are fundamental yet disconnected components in autonomous driving. Prediction models forecast surrounding agent motion under unknown intentions, producing multimodal distributions, while planning assumes known ego objectives and generates deterministic trajectories. This mismatch creates a critical bottleneck: prediction lacks supervision for agent intentions, while planning requires this information. Existing prediction models, despite strong benchmarking performance, often remain disconnected from planning constraints such as collision avoidance and dynamic feasibility. We introduce Plan TRansformer (PTR), a unified Gaussian Mixture Transformer framework integrating goal-conditioned prediction, dynamic feasibility, interaction awareness, and lane-level topology reasoning. A teacher-student training strategy progressively masks surrounding agent commands during training to align with inference conditions where agent intentions are unavailable. PTR achieves 4.3%/3.5% improvement in marginal/joint mAP compared to the baseline Motion Transformer (MTR) and 15.5% planning error reduction at 5s horizon compared to GameFormer. The architecture-agnostic design enables application to diverse Transformer-based prediction models. Project Website: https://github.com/SelzerConst/PlanTRansformer

Related papers

• Improving Consistency in Vehicle Trajectory Prediction Through Preference Optimization [4.506411269983418]
  Trajectory prediction is an essential step in the pipeline of an autonomous vehicle.<n>Current deep-learning-based trajectory prediction models can achieve excellent accuracy on public datasets.<n>This work fine-tunes trajectory prediction models in multi-agent settings using preference optimization.
  arXiv  Detail & Related papers  (2025-07-03T07:59:49Z)
• Planning with Adaptive World Models for Autonomous Driving [50.4439896514353]
  We present nuPlan, a real-world motion planning benchmark that captures multi-agent interactions.<n>We learn to model such unique behaviors with BehaviorNet, a graph convolutional neural network (GCNN)<n>We also present AdaptiveDriver, a model-predictive control (MPC) based planner that unrolls different world models conditioned on BehaviorNet's predictions.
  arXiv  Detail & Related papers  (2024-06-15T18:53:45Z)
• Interactive Joint Planning for Autonomous Vehicles [19.479300967537675]
  In interactive driving scenarios, the actions of one agent greatly influences those of its neighbors. We present Interactive Joint Planning (IJP) that bridges MPC with learned prediction models. IJP significantly outperforms the baselines that are either without joint optimization or running sampling-based planning.
  arXiv  Detail & Related papers  (2023-10-27T17:48:25Z)
• AdvDO: Realistic Adversarial Attacks for Trajectory Prediction [87.96767885419423]
  Trajectory prediction is essential for autonomous vehicles to plan correct and safe driving behaviors. We devise an optimization-based adversarial attack framework to generate realistic adversarial trajectories. Our attack can lead an AV to drive off road or collide into other vehicles in simulation.
  arXiv  Detail & Related papers  (2022-09-19T03:34:59Z)
• ScePT: Scene-consistent, Policy-based Trajectory Predictions for Planning [32.71073060698739]
  Trajectory prediction is critical for autonomous systems that share environments with uncontrolled agents. We present ScePT, a policy planning-based trajectory prediction model. It explicitly enforces scene consistency and learns an agent interaction policy that can be used for conditional prediction.
  arXiv  Detail & Related papers  (2022-06-18T00:00:02Z)
• Control-Aware Prediction Objectives for Autonomous Driving [78.19515972466063]
  We present control-aware prediction objectives (CAPOs) to evaluate the downstream effect of predictions on control without requiring the planner be differentiable. We propose two types of importance weights that weight the predictive likelihood: one using an attention model between agents, and another based on control variation when exchanging predicted trajectories for ground truth trajectories.
  arXiv  Detail & Related papers  (2022-04-28T07:37:21Z)
• Deep Interactive Motion Prediction and Planning: Playing Games with Motion Prediction Models [162.21629604674388]
  This work presents a game-theoretic Model Predictive Controller (MPC) that uses a novel interactive multi-agent neural network policy as part of its predictive model. Fundamental to the success of our method is the design of a novel multi-agent policy network that can steer a vehicle given the state of the surrounding agents and the map information.
  arXiv  Detail & Related papers  (2022-04-05T17:58:18Z)
• Reactive motion planning with probabilistic safety guarantees [27.91467018272684]
  This paper considers the problem of motion planning in environments with multiple uncontrolled agents. A predictive model of the uncontrolled agents is trained to predict all possible trajectories within a short horizon based on the scenario. The proposed approach is demonstrated in simulation in a scenario emulating autonomous highway driving.
  arXiv  Detail & Related papers  (2020-11-06T20:37:15Z)
• The Importance of Prior Knowledge in Precise Multimodal Prediction [71.74884391209955]
  Roads have well defined geometries, topologies, and traffic rules. In this paper we propose to incorporate structured priors as a loss function. We demonstrate the effectiveness of our approach on real-world self-driving datasets.
  arXiv  Detail & Related papers  (2020-06-04T03:56:11Z)
• PiP: Planning-informed Trajectory Prediction for Autonomous Driving [69.41885900996589]
  We propose planning-informed trajectory prediction (PiP) to tackle the prediction problem in the multi-agent setting. By informing the prediction process with the planning of ego vehicle, our method achieves the state-of-the-art performance of multi-agent forecasting on highway datasets.
  arXiv  Detail & Related papers  (2020-03-25T16:09:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.