ParkingTransformer: LLM-Enhanced End-to-End Trajectory Planning for Autonomous Parking

• URL: http://arxiv.org/abs/2606.17082v1
• Date: Fri, 12 Jun 2026 05:52:01 GMT
• Title: ParkingTransformer: LLM-Enhanced End-to-End Trajectory Planning for Autonomous Parking
• Abstract summary: ParkingTransformer is a novel framework that leverages multi-view perception and the scene understanding capability of Large Language Models (LLMs)<n>To compensate for the inadequate spatial reasoning ability of LLMs, we introduce 3D positional encoding to explicitly inject spatial geometric awareness.<n>Our method achieves a driving score of 61.32 in CARLA simulator and an average success rate of 88.70% in real-world experiments.
• Score: 13.901233386612722
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: End-to-end autonomous parking has emerged as a critical task within the realm of autonomous driving. However, existing methods suffer from black-box characteristics, lacking high-level semantic understanding and interpretability, which impedes the realization of seamless long-distance autonomous parking from the road to the target spot. To address these limitations, we propose ParkingTransformer, a novel framework that leverages multi-view perception and the scene understanding capability of Large Language Models (LLMs). By combining trajectory queries with LLMs implicit state features, our method interacts directly with historical information and raw sensor data to output planning trajectories, eliminating the need for dense Bird's-View (BEV) representations. To compensate for the inadequate spatial reasoning ability of LLMs, we introduce 3D positional encoding to explicitly inject spatial geometric awareness. Furthermore, a fixed-window streaming mechanism is designed for historical information processing, significantly improving long-term temporal processing efficiency and inference speed. Additionally, a coarse-to-fine decoding strategy is employed to progressively enhance trajectory precision. Extensive closed-loop experiments are conducted on the CARLA simulator and real-world vehicle platforms. The results demonstrate that our method achieves a driving score of 61.32 in CARLA simulator and an average success rate of 88.70% in real-world experiments, validating the feasibility and effectiveness of the proposed algorithms.

Related papers

• Large Language Model Enhanced Differentiable Trajectory Planning for IoT-Enabled Autonomous Driving [26.33691640001489]
  This paper proposes a large language model (LLM) enhanced differentiable trajectory planning framework for IoT-enabled autonomous driving.<n>We introduce a surrounding agent centric data augmentation strategy to reorganize sur rounding agent trajectories as additional planning supervision.<n>We also design a complexity-aware asyn chronous LLM-based semantic enhancement module to extract scene-related high-level semantic features.
  arXiv  Detail & Related papers  (2026-07-11T18:43:14Z)
• SunnyParking: Multi-Shot Trajectory Generation and Motion State Awareness for Human-like Parking [24.480545421791234]
  We propose a novel dual-branch E2E architecture that achieves motion state awareness by jointly predicting spatial trajectories and discrete motion state sequences.<n> Experimental results demonstrate that our framework generates more robust and human-like trajectories in complex multi-shot parking scenarios.<n>We open-source a new parking dataset of the CARLA simulator, specifically designed to evaluate full prediction capabilities under complex maneuvers.
  arXiv  Detail & Related papers  (2026-02-25T08:35:58Z)
• LLM-Driven Scenario-Aware Planning for Autonomous Driving [35.75515733294211]
  Hybrid planner switching framework (HPSF) for autonomous driving needs to reconcile high-speed driving efficiency with safe maneuvering in dense traffic.<n>Existing HPSF methods often fail to make reliable mode transitions or sustain efficient driving in congested environments.<n>This paper proposes a large language model (LLM) driven, adaptive planning method, which switches between high-speed driving in low-complexity scenes and precise driving in high-complexity scenes.
  arXiv  Detail & Related papers  (2026-01-29T15:42:13Z)
• A Vision-Language-Action Model with Visual Prompt for OFF-Road Autonomous Driving [14.778856339414455]
  Efficient trajectory planning in off-road terrains presents a formidable challenge for autonomous vehicles.<n>This paper proposes OFF-EMMA, a novel end-to-end multimodal framework to overcome the deficiencies of insufficient spatial perception and unstable reasoning.<n> Experimental results on the RELLIS-3D off-road dataset demonstrate that OFF-EMMA significantly outperforms existing methods.
  arXiv  Detail & Related papers  (2026-01-07T02:08:18Z)
• ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving [49.07731497951963]
  ReCogDrive is a novel Reinforced Cognitive framework for end-to-end autonomous driving.<n>We introduce a hierarchical data pipeline that mimics the sequential cognitive process of human drivers.<n>We then address the language-action mismatch by injecting the VLM's learned driving priors into a diffusion planner.
  arXiv  Detail & Related papers  (2025-06-09T03:14:04Z)
• SOLVE: Synergy of Language-Vision and End-to-End Networks for Autonomous Driving [51.47621083057114]
  SOLVE is an innovative framework that synergizes Vision-Language Models with end-to-end (E2E) models to enhance autonomous vehicle planning.<n>Our approach emphasizes knowledge sharing at the feature level through a shared visual encoder, enabling comprehensive interaction between VLM and E2E components.
  arXiv  Detail & Related papers  (2025-05-22T15:44:30Z)
• Tracking Meets Large Multimodal Models for Driving Scenario Understanding [76.71815464110153]
  Large Multimodal Models (LMMs) have recently gained prominence in autonomous driving research.<n>We propose to integrate tracking information as an additional input to recover 3D spatial and temporal details.<n>We introduce a novel approach for embedding this tracking information into LMMs to enhance their understanding of driving scenarios.
  arXiv  Detail & Related papers  (2025-03-18T17:59:12Z)
• RAD: Retrieval-Augmented Decision-Making of Meta-Actions with Vision-Language Models in Autonomous Driving [10.984203470464687]
  Vision-language models (VLMs) often suffer from limitations such as inadequate spatial perception and hallucination.<n>We propose a retrieval-augmented decision-making (RAD) framework to enhance VLMs' capabilities to reliably generate meta-actions in autonomous driving scenes.<n>We fine-tune VLMs on a dataset derived from the NuScenes dataset to enhance their spatial perception and bird's-eye view image comprehension capabilities.
  arXiv  Detail & Related papers  (2025-03-18T03:25:57Z)
• BEVDriver: Leveraging BEV Maps in LLMs for Robust Closed-Loop Driving [0.5735035463793009]
  Large Language Models (LLMs) possess reasoning capabilities and natural language understanding.<n>We introduce BEVDriver, an LLM-based model for end-to-end closed-loop driving in CARLA.<n>On the LangAuto benchmark, our model reaches up to 18.9% higher performance on the Driving Score compared to SoTA methods.
  arXiv  Detail & Related papers  (2025-03-05T00:27:32Z)
• DiFSD: Ego-Centric Fully Sparse Paradigm with Uncertainty Denoising and Iterative Refinement for Efficient End-to-End Self-Driving [55.53171248839489]
  We propose an ego-centric fully sparse paradigm, named DiFSD, for end-to-end self-driving.<n>Specifically, DiFSD mainly consists of sparse perception, hierarchical interaction and iterative motion planner.<n>Experiments conducted on nuScenes and Bench2Drive datasets demonstrate the superior planning performance and great efficiency of DiFSD.
  arXiv  Detail & Related papers  (2024-09-15T15:55:24Z)
• LLM-A*: Large Language Model Enhanced Incremental Heuristic Search on Path Planning [91.95362946266577]
  Path planning is a fundamental scientific problem in robotics and autonomous navigation.<n>Traditional algorithms like A* and its variants are capable of ensuring path validity but suffer from significant computational and memory inefficiencies as the state space grows.<n>We propose a new LLM based route planning method that synergistically combines the precise pathfinding capabilities of A* with the global reasoning capability of LLMs.<n>This hybrid approach aims to enhance pathfinding efficiency in terms of time and space complexity while maintaining the integrity of path validity, especially in large-scale scenarios.
  arXiv  Detail & Related papers  (2024-06-20T01:24:30Z)
• Unsupervised Domain Adaptation for Self-Driving from Past Traversal Features [69.47588461101925]
  We propose a method to adapt 3D object detectors to new driving environments. Our approach enhances LiDAR-based detection models using spatial quantized historical features. Experiments on real-world datasets demonstrate significant improvements.
  arXiv  Detail & Related papers  (2023-09-21T15:00:31Z)
• Multi-Modal Fusion Transformer for End-to-End Autonomous Driving [59.60483620730437]
  We propose TransFuser, a novel Multi-Modal Fusion Transformer, to integrate image and LiDAR representations using attention. Our approach achieves state-of-the-art driving performance while reducing collisions by 76% compared to geometry-based fusion.
  arXiv  Detail & Related papers  (2021-04-19T11:48:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.