Cognitive Dual-Process Planning for Autonomous Driving with Structured Scene Knowledge and Verifiable Reasoning-Action Consistency

• URL: http://arxiv.org/abs/2607.19194v2
• Date: Wed, 22 Jul 2026 03:53:48 GMT
• Title: Cognitive Dual-Process Planning for Autonomous Driving with Structured Scene Knowledge and Verifiable Reasoning-Action Consistency
• Abstract summary: Vision-language models (VLMs) can make intermediate reasoning explicit, but their use in deployed planners is constrained by costly structured supervision.<n>We present a cognitive dual-process planning framework that represents planning-relevant scene knowledge in a machine-parsable structured chain-of-thought.<n>We show how explicit scene knowledge can be operationalized through adaptive reasoning and rule-based verification to support high-level VLM planning decisions.
• Score: 15.031457072334662
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: High-level planning for autonomous driving is a knowledge-intensive engineering decision task that requires accurate scene understanding, timely inference, and internally consistent action selection. Vision-language models (VLMs) can make intermediate reasoning explicit, but their use in deployed planners is constrained by costly structured supervision, unnecessary reasoning in routine scenes, and possible inconsistencies between generated rationales and driving actions. We present a cognitive dual-process planning framework that represents planning-relevant scene knowledge in a machine-parsable structured chain-of-thought (S-CoT) schema. An automated data engine integrates perception foundation models, critical-path filtering, and an expert VLM to generate S-CoT supervision without manual annotation of individual rationales. A lightweight visual Arbiter estimates scene complexity from multilevel vision-encoder features before language decoding and routes each input to either fast meta-action prediction or slow structured reasoning. For slow-path outputs, a deterministic rule-based validator checks whether the parsed S-CoT fields are consistent with the final meta-action and provides verifiable rewards for Group Relative Policy Optimization (GRPO). In a 195-scene manual audit, the generated annotations achieve 91.8\% CoT accuracy and a 98.5\% Logical Consistency Score (LCS). On 574 manually verified NAVSIM test samples, the planner achieves 80.14\% planning accuracy and 97.20\% LCS while reducing average latency by 17.39\% relative to applying slow reasoning to every scene. Evaluation on external long-tail subsets further identifies conditions under which routing and planning performance degrade. Together, these results show how explicit scene knowledge can be operationalized through adaptive reasoning and rule-based verification to support high-level VLM planning decisions.

Related papers

• FactorDrive: Adaptive Multi-Step Reasoning Driven by Planning-Critical Factors for End-to-End Autonomous Driving [8.241982806652256]
  We propose FactorDrive, an end-to-end framework for adaptive multi-step reasoning driven by planning-critical factors (PCFs)<n>PCF-CoT dataset grounds planning reasoning in trajectory-relevant spatial-physical evidence.<n>QS-GRPO guides Monte Carlo Tree Search (MCTS) with trajectory-level planning rewards to discover reasoning paths with higher planning quality.
  arXiv  Detail & Related papers  (2026-08-10T13:26:09Z)
• CLEAR: Cognition and Latent Evaluation for Adaptive Routing in End-to-End Autonomous Driving [19.474428775260034]
  We propose CLEAR, a framework that combines ultra-fast generative planning with deep semantic reasoning.<n> CLEAR employs Drive-JEPA as the visual encoder and replaces the multi-step denoising chain with a single-step conditional drift in a VAE latent space.<n>On the NAVSIM v1 benchmark, CLEAR achieves a state-of-the-art PDMS of 93.7.
  arXiv  Detail & Related papers  (2026-06-04T14:32:10Z)
• Steered Generation via Gradient-Based Optimization on Sparse Query Features [1.587869707099313]
  Prototype-Based Sparse Steering is a framework that applies Sparse Autoencoders specifically to query activations.<n>We demonstrate that sparse query representations provide the necessary disentanglement for unified, interpretable control over both logical planning and stylistic nuance.
  arXiv  Detail & Related papers  (2026-05-21T21:13:14Z)
• VLA-ATTC: Adaptive Test-Time Compute for VLA Models with Relative Action Critic Model [54.35791816657227]
  We introduce textbfVLA-ATTC', a framework that endows VLA models with adaptive test-time compute.<n>VLA-ATTC employs an uncertainty-based cognitive clutch'' to dynamically transition from reflexive execution to a TTC deliberation phase.<n>On the LIBERO-LONG benchmark, VLA-ATTC reduces the failure rate of the SOTA model PI0.5 by over 50%.
  arXiv  Detail & Related papers  (2026-05-02T02:13:11Z)
• OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models [69.2503510410147]
  We present a unified autonomous driving framework built upon a pretrained VLM.<n>We demonstrate that pretrained VLM attention exhibits strong transferability beyond pure language modeling.<n>Experiments on end-to-end autonomous driving benchmarks demonstrate state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-04-20T07:50:00Z)
• Open-Ended Instruction Realization with LLM-Enabled Multi-Planner Scheduling in Autonomous Vehicles [40.195332742819865]
  This study proposes an instruction-realization framework that leverages a large language model (LLM) to interpret instructions.<n>It generates executable scripts that schedule multiple model predictive control (MPC)-based motion planners based on real-time feedback, and converts planned trajectories into control signals.
  arXiv  Detail & Related papers  (2026-04-09T09:32:21Z)
• Bridging Large-Model Reasoning and Real-Time Control via Agentic Fast-Slow Planning [42.15812524999639]
  Agentic Fast-Slow Planning is a hierarchical framework that decouples perception, reasoning, planning, and control across natural timescales.<n>We show that Agentic Fast-Slow Planning improves robustness under perturbations, reducing lateral deviation by up to 45% and completion time by over 12% compared to pure MPC and an A*-guided MPC baseline.
  arXiv  Detail & Related papers  (2026-04-02T06:34:29Z)
• SGDrive: Scene-to-Goal Hierarchical World Cognition for Autonomous Driving [52.02379432801349]
  We propose SGDrive, a novel framework that structures the VLM's representation learning around driving-specific knowledge hierarchies.<n>Built upon a pre-trained VLM backbone, SGDrive decomposes driving understanding into a scene-agent-goal hierarchy that mirrors human driving cognition.
  arXiv  Detail & Related papers  (2026-01-09T08:55:42Z)
• Foundation Models for Logistics: Toward Certifiable, Conversational Planning Interfaces [59.80143393787701]
  Large language models (LLMs) can handle uncertainty and promise to accelerate replanning while lowering the barrier to entry.<n>We introduce a neurosymbolic framework that pairs the accessibility of natural-language dialogue with verifiable guarantees on goal interpretation.<n>A lightweight model, fine-tuned on just 100 uncertainty-filtered examples, surpasses the zero-shot performance of GPT-4.1 while cutting inference latency by nearly 50%.
  arXiv  Detail & Related papers  (2025-07-15T14:24:01Z)
• Control-Aware Prediction Objectives for Autonomous Driving [78.19515972466063]
  We present control-aware prediction objectives (CAPOs) to evaluate the downstream effect of predictions on control without requiring the planner be differentiable. We propose two types of importance weights that weight the predictive likelihood: one using an attention model between agents, and another based on control variation when exchanging predicted trajectories for ground truth trajectories.
  arXiv  Detail & Related papers  (2022-04-28T07:37:21Z)
• An Information Bottleneck Approach for Controlling Conciseness in Rationale Extraction [84.49035467829819]
  We show that it is possible to better manage this trade-off by optimizing a bound on the Information Bottleneck (IB) objective. Our fully unsupervised approach jointly learns an explainer that predicts sparse binary masks over sentences, and an end-task predictor that considers only the extracted rationale.
  arXiv  Detail & Related papers  (2020-05-01T23:26:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.