Depth-Wise Probing and Pruning of the Planning Token in a Driving Vision-Language-Action Model

• URL: http://arxiv.org/abs/2608.07361v1
• Date: Fri, 07 Aug 2026 16:02:34 GMT
• Title: Depth-Wise Probing and Pruning of the Planning Token in a Driving Vision-Language-Action Model
• Abstract summary: We study a representative driving VLA whose entire plan is carried by a single planning token that a generative planner decodes into a trajectory.<n>We measure two signals: the linear decodability of the navigation command and trajectory compatibility with the frozen native planner.
• Score: 10.925063608741844
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-language-action (VLA) models route driving decisions through a deep language model, but it is unclear how much of that depth the action itself requires. We study a representative driving VLA whose entire plan is carried by a single planning token that a generative planner decodes into a trajectory. Borrowing the planner as a trajectory-space logit lens, we decode the planning token from every one of the 32 decoder layers and measure two signals: the linear decodability of the navigation command and trajectory compatibility with the frozen native planner. Our diagnostic shows that semantic intent is linearly decodable early: command-probe accuracy reaches 97.7\% after the first decoder layer, compared with 16.7\% chance. In contrast, compatibility with the frozen native planner improves gradually across depth, with open-loop Avg-L2 reaching its minimum of 2.11\,m only at the final layer. Learned readouts from the first layer recover much of this gap, indicating that planning information is already present early but is not yet represented in the format expected by the deployed planner. Ranking decoder layers by the angular deviation they induce in the planning token permits removal of 8 of 32 layers within an approximately 5\% relative open-loop error increase and yields a measured 1.33$\times$ decoder speedup. At the evaluated sample size, no family-specific degradation is statistically resolved. These findings are limited to the evaluated ORION checkpoint and Bench2Drive setup.

Related papers

• LePlanner: An Iterative Amortized Controller For World Models [0.0]
  We propose LePlanner, an amortized iterative controller that learns to construct and refine latent action sequences through a frozen world-model predictor.<n>LePlanner is trained with an arrival-and-hold objective that encourages the controller to reach the goal at the earliest feasible horizon and remain there.<n>It achieves success rates of 98% on PushT, 100% on Reacher, 100% on TwoRooms, and 92% on the OGBench Cube task.
  arXiv  Detail & Related papers  (2026-09-12T10:01:21Z)
• Explore From Sketch: Accelerating UAV Exploration in Large-scale Environments with Prior Maps [20.559950853029]
  This paper presents a novel exploration framework that leverages sparse, unaligned, and even discrepant 2D prior maps for LiDAR-based UAV exploration.<n> Benchmark evaluations reveal up to 34.2% improvement in exploration efficiency and 37.9% reduction in flight distance compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-06-10T06:30:29Z)
• When is Your LLM Steerable? [56.656180566692946]
  Activation steering offers a lightweight approach to control language models' behavior at inference time.<n>Finding the regime and boundaries of successful steering typically requires expensive grid searches.<n>We investigate whether steerability can be predicted from the model's internal states at the beginning of the generation process.
  arXiv  Detail & Related papers  (2026-06-10T02:55:34Z)
• LLaViDA: A Large Language Vision Driving Assistant for Explicit Reasoning and Enhanced Trajectory Planning [28.59507336524504]
  Trajectory planning is a fundamental yet challenging component of autonomous driving.<n>We propose LLaViDA, a Large Language Vision Driving Assistant that leverages a Vision-Language Model (VLM) for object motion prediction.<n>On the NuScenes benchmark, LLaViDA surpasses state-of-the-art end-to-end and other recent VLM/LLM-based baselines in open-loop trajectory planning task.
  arXiv  Detail & Related papers  (2025-12-20T04:38:35Z)
• Blurred Encoding for Trajectory Representation Learning [56.642548506419]
  Tray representation learning (TRL) maps trajectories to vector embeddings and facilitates tasks such as trajectory classification and similarity search.<n>State-of-the-art (SOTA) TRL methods transform raw GPS trajectories to grid or road trajectories to capture high-level travel semantics.<n>But they lose fine-grained spatial-temporal details as multiple GPS points are grouped into a single grid cell or road segment.<n>We propose the BLUrred jector method, dubbed BLUE, which gradually reduces the precision of GPS coordinates to create hierarchical patches.
  arXiv  Detail & Related papers  (2025-11-12T03:51:57Z)
• GPD-1: Generative Pre-training for Driving [77.06803277735132]
  We propose a unified Generative Pre-training for Driving (GPD-1) model to accomplish all these tasks.<n>We represent each scene with ego, agent, and map tokens and formulate autonomous driving as a unified token generation problem.<n>Our GPD-1 successfully generalizes to various tasks without finetuning, including scene generation, traffic simulation, closed-loop simulation, map prediction, and motion planning.
  arXiv  Detail & Related papers  (2024-12-11T18:59:51Z)
• PRET: Planning with Directed Fidelity Trajectory for Vision and Language Navigation [30.710806048991923]
  Vision and language navigation is a task that requires an agent to navigate according to a natural language instruction. Recent methods predict sub-goals on constructed topology map at each step to enable long-term action planning. We propose an alternative method that facilitates navigation planning by considering the alignment between instructions and directed fidelity trajectories.
  arXiv  Detail & Related papers  (2024-07-16T08:22:18Z)
• Affordances-Oriented Planning using Foundation Models for Continuous Vision-Language Navigation [64.84996994779443]
  We propose a novel Affordances-Oriented Planner for continuous vision-language navigation (VLN) task. Our AO-Planner integrates various foundation models to achieve affordances-oriented low-level motion planning and high-level decision-making. Experiments on the challenging R2R-CE and RxR-CE datasets show that AO-Planner achieves state-of-the-art zero-shot performance.
  arXiv  Detail & Related papers  (2024-07-08T12:52:46Z)
• V-DETR: DETR with Vertex Relative Position Encoding for 3D Object Detection [73.37781484123536]
  We introduce a highly performant 3D object detector for point clouds using the DETR framework. To address the limitation, we introduce a novel 3D Relative Position (3DV-RPE) method. We show exceptional results on the challenging ScanNetV2 benchmark.
  arXiv  Detail & Related papers  (2023-08-08T17:14:14Z)
• Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenes [38.43491956142818]
  Planning task involves predicting the trajectory of the ego vehicle based on inputs from both internal intention and the external environment. Most existing works evaluate their performance on the nuScenes dataset using the L2 error and collision rate between the predicted trajectories and the ground truth. In this paper, we reevaluate these existing evaluation metrics and explore whether they accurately measure the superiority of different methods. Our simple method achieves similar end-to-end planning performance on the nuScenes dataset with other perception-based methods, reducing the average L2 error by about 20%.
  arXiv  Detail & Related papers  (2023-05-17T17:59:11Z)
• 1st Place Solutions for RxR-Habitat Vision-and-Language Navigation Competition (CVPR 2022) [28.5740809300599]
  We present a modular plan-and-control approach for the problem of Vision-and-Language Navigation in Continuous Environments (VLN-CE) Our model consists of three modules: the candidate waypoints predictor (CWP), the history enhanced planner and the tryout controller. Our model won the RxR-Habitat Competition 2022, with 48% and 90% relative improvements over existing methods on NDTW and SR metrics respectively.
  arXiv  Detail & Related papers  (2022-06-23T10:36:53Z)
• Long-Horizon Visual Planning with Goal-Conditioned Hierarchical Predictors [124.30562402952319]
  The ability to predict and plan into the future is fundamental for agents acting in the world. Current learning approaches for visual prediction and planning fail on long-horizon tasks. We propose a framework for visual prediction and planning that is able to overcome both of these limitations.
  arXiv  Detail & Related papers  (2020-06-23T17:58:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.