Align2Act: Instruction-Tuned Models for Human-Aligned Autonomous Driving

• URL: http://arxiv.org/abs/2510.10503v1
• Date: Sun, 12 Oct 2025 08:50:34 GMT
• Title: Align2Act: Instruction-Tuned Models for Human-Aligned Autonomous Driving
• Authors: Kanishkha Jaisankar, Sunidhi Tandel,
• Abstract summary: We propose Align2Act, a motion planning framework that transforms instruction-tuned language models into interpretable planners aligned with human behavior.<n>By fine-tuning LLaMA-2-7B with LoRA on one million scenarios from the nuPlan dataset, our method achieves an open-loop score of 85.17 and closed-loop scores of 70.31 and 66.96 on Test14-random.<n>Unlike prior work focused on synthetic or open-loop settings, we demonstrate improved planning quality and human-likeness on the real-world nuPlan closed-loop benchmark.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Motion planning in complex scenarios is a core challenge in autonomous driving. Conventional methods apply predefined rules or learn from driving data to generate trajectories, while recent approaches leverage large language models (LLMs) for decision-making. However, it remains unclear whether LLMs truly capture human driving logic. We propose Align2Act, a motion planning framework that transforms instruction-tuned LLMs into interpretable planners aligned with human behavior. We derive structured driving instructions based on human reasoning patterns (e.g., anticipate hazards, yield at intersections) and traffic rules (e.g., stop at red lights, maintain lane boundaries). Our Align2ActChain module guides step-by-step reasoning to produce both an interpretable rationale and a safe trajectory. By fine-tuning LLaMA-2-7B with LoRA on one million scenarios from the nuPlan dataset, our method achieves an open-loop score of 85.17 and closed-loop scores of 70.31 (non-reactive) and 66.96 (reactive) on Test14-random. Unlike prior work focused on synthetic or open-loop settings, we demonstrate improved planning quality and human-likeness on the real-world nuPlan closed-loop benchmark. Ablation studies confirm that structured reasoning significantly improves performance over baseline LLM planners.

Related papers

• MindDrive: A Vision-Language-Action Model for Autonomous Driving via Online Reinforcement Learning [51.20229133553804]
  Current Vision-Language-Action (VLA) paradigms in autonomous driving primarily rely on Imitation Learning (IL)<n>Online Reinforcement Learning offers a promising pathway to address these issues through trial-and-error learning.<n>We propose MindDrive, a VLA framework comprising a large language model (LLM) with two distinct sets of LoRA parameters.<n>By feeding trajectory-level rewards back into the reasoning space, MindDrive enables trial-and-error learning over a finite set of discrete linguistic driving decisions.
  arXiv  Detail & Related papers  (2025-12-15T18:31:32Z)
• ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving [49.07731497951963]
  ReCogDrive is a novel Reinforced Cognitive framework for end-to-end autonomous driving.<n>We introduce a hierarchical data pipeline that mimics the sequential cognitive process of human drivers.<n>We then address the language-action mismatch by injecting the VLM's learned driving priors into a diffusion planner.
  arXiv  Detail & Related papers  (2025-06-09T03:14:04Z)
• Plan-R1: Safe and Feasible Trajectory Planning as Language Modeling [74.41886258801209]
  We propose a two-stage trajectory planning framework that decouples principle alignment from behavior learning.<n>Plan-R1 significantly improves planning safety and feasibility, achieving state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-05-23T09:22:19Z)
• LiloDriver: A Lifelong Learning Framework for Closed-loop Motion Planning in Long-tail Autonomous Driving Scenarios [23.913788819453796]
  LiloDriver is a lifelong learning framework for closed-loop motion planning in long-tail autonomous driving scenarios.<n>It features a four-stage architecture including perception, scene encoding, memory-based strategy refinement, and LLM-guided reasoning.<n>Our results highlight the effectiveness of combining structured memory and LLM reasoning to enable scalable, human-like motion planning in real-world autonomous driving.
  arXiv  Detail & Related papers  (2025-05-22T18:33:08Z)
• A Framework for Learning Scoring Rules in Autonomous Driving Planning Systems [2.4578723416255754]
  FLoRA is a framework that learns interpretable scoring rules represented in temporal logic.<n>Our approach effectively learns to evaluate driving behavior even though the training data only contains positive examples.<n> Evaluations in closed-loop planning simulations demonstrate that our learned scoring rules outperform existing techniques.
  arXiv  Detail & Related papers  (2025-02-17T02:06:57Z)
• Diffusion-Based Planning for Autonomous Driving with Flexible Guidance [19.204115959760788]
  We propose a novel transformer-based Diffusion Planner for closed-loop planning.<n>Our model supports joint modeling of both prediction and planning tasks.<n>It achieves state-of-the-art closed-loop performance with robust transferability in diverse driving styles.
  arXiv  Detail & Related papers  (2025-01-26T15:49:50Z)
• Instruct Large Language Models to Drive like Humans [33.219883052634614]
  We propose an InstructDriver method to transform large language models into motion planners. We derive driving instruction data based on human logic. We then employ an interpretable InstructChain module to further reason the final planning.
  arXiv  Detail & Related papers  (2024-06-11T14:24:45Z)
• LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoning [65.86754998249224]
  We develop a novel hybrid planner that leverages a conventional rule-based planner in conjunction with an LLM-based planner. Our approach navigates complex scenarios which existing planners struggle with, produces well-reasoned outputs while also remaining grounded through working alongside the rule-based approach.
  arXiv  Detail & Related papers  (2023-12-30T02:53:45Z)
• How To Not Drive: Learning Driving Constraints from Demonstration [0.0]
  We propose a new scheme to learn motion planning constraints from human driving trajectories. The behavioral planning is responsible for high-level decision making required to follow traffic rules. The motion planner role is to generate feasible, safe trajectories for a self-driving vehicle to follow.
  arXiv  Detail & Related papers  (2021-10-01T20:47:04Z)
• Learning to drive from a world on rails [78.28647825246472]
  We learn an interactive vision-based driving policy from pre-recorded driving logs via a model-based approach. A forward model of the world supervises a driving policy that predicts the outcome of any potential driving trajectory. Our method ranks first on the CARLA leaderboard, attaining a 25% higher driving score while using 40 times less data.
  arXiv  Detail & Related papers  (2021-05-03T05:55:30Z)
• Deep Structured Reactive Planning [94.92994828905984]
  We propose a novel data-driven, reactive planning objective for self-driving vehicles. We show that our model outperforms a non-reactive variant in successfully completing highly complex maneuvers.
  arXiv  Detail & Related papers  (2021-01-18T01:43:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.