Language-Driven Cost Optimization for Autonomous Driving

• URL: http://arxiv.org/abs/2606.10974v1
• Date: Tue, 09 Jun 2026 15:13:30 GMT
• Title: Language-Driven Cost Optimization for Autonomous Driving
• Abstract summary: This work presents a language-driven framework for adaptive cost design in autonomous driving.<n>A Large Language Model (LLM) interprets structured scenario descriptions and natural language user queries.<n>The system incorporates a human-in-the-loop validation stage in which the proposed behavioral changes are described in non-technical language.
• Score: 15.884773782381346
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The driving behavior of autonomous vehicles is typically governed by the cost function of their motion planner, which encodes objectives such as speed tracking, smoothness, lane keeping, and collision avoidance. However, tuning the parameters that shape this cost function is a challenging task that requires technical expertise, limiting the vehicle's ability to adapt to evolving traffic scenarios or end-user preferences. This work presents a language-driven framework for adaptive cost design in autonomous driving. A Large Language Model (LLM) interprets structured scenario descriptions and natural language user queries to generate the parameters applied to a risk-aware Model Predictive Path Integral (MPPI) controller. The system incorporates a human-in-the-loop validation stage in which the proposed behavioral changes are described in non-technical language and confirmed prior to deployment. Users may additionally provide feedback either before or after deployment, enabling iterative refinement of the vehicle's motion behavior. The framework is evaluated across multiple queries in realistic driving scenarios to assess its effectiveness. Simulation results demonstrate that the method successfully induces behavioral changes that align with the intended requirements in an intuitive manner, thereby bridging the gap between intelligent vehicle control systems and end users.

Related papers

• CarPLAN: Context-Adaptive and Robust Planning with Dynamic Scene Awareness for Autonomous Driving [24.617191209246226]
  CarPLAN is a novel IL-based motion planning framework that explicitly enhances driving context understanding.<n> Displacement-Aware Predictive generalization (DPE) improves the model's spatial awareness.<n> Context-adaptive Multi-Expert Decoder (CMD) improves the model's ability to handle diverse driving contexts.
  arXiv  Detail & Related papers  (2026-03-13T03:22:32Z)
• ImagiDrive: A Unified Imagination-and-Planning Framework for Autonomous Driving [64.12414815634847]
  Vision-Language Models (VLMs) and Driving World Models (DWMs) have independently emerged as powerful recipes addressing different aspects of this challenge.<n>We propose ImagiDrive, a novel end-to-end autonomous driving framework that integrates a VLM-based driving agent with a DWM-based scene imaginer.
  arXiv  Detail & Related papers  (2025-08-15T12:06:55Z)
• Test Automation for Interactive Scenarios via Promptable Traffic Simulation [48.240394447516664]
  We introduce an automated method to generate realistic and safety-critical human behaviors for AV planner evaluation in interactive scenarios.<n>We parameterize complex human behaviors using low-dimensional goal positions, which are then fed into a promptable traffic simulator, ProSim.<n>To automate test generation, we introduce a prompt generation module that explores the goal domain and efficiently identifies safety-critical behaviors using Bayesian optimization.
  arXiv  Detail & Related papers  (2025-06-01T22:29:32Z)
• LANGTRAJ: Diffusion Model and Dataset for Language-Conditioned Trajectory Simulation [102.1527101235251]
  LangTraj is a language-conditioned scene-diffusion model that simulates the joint behavior of all agents in traffic scenarios.<n>By conditioning on natural language inputs, LangTraj provides flexible and intuitive control over interactive behaviors.<n>LangTraj demonstrates strong performance in realism, language controllability, and language-conditioned safety-critical simulation.
  arXiv  Detail & Related papers  (2025-04-15T17:14:06Z)
• A Framework for Learning Scoring Rules in Autonomous Driving Planning Systems [2.4578723416255754]
  FLoRA is a framework that learns interpretable scoring rules represented in temporal logic.<n>Our approach effectively learns to evaluate driving behavior even though the training data only contains positive examples.<n> Evaluations in closed-loop planning simulations demonstrate that our learned scoring rules outperform existing techniques.
  arXiv  Detail & Related papers  (2025-02-17T02:06:57Z)
• EditFollower: Tunable Car Following Models for Customizable Adaptive Cruise Control Systems [28.263763430300504]
  We propose a data-driven car-following model that allows for adjusting driving discourtesy levels. Our model provides valuable insights for the development of ACC systems that take into account drivers' social preferences.
  arXiv  Detail & Related papers  (2024-06-23T15:04:07Z)
• TLControl: Trajectory and Language Control for Human Motion Synthesis [68.09806223962323]
  We present TLControl, a novel method for realistic human motion synthesis. It incorporates both low-level Trajectory and high-level Language semantics controls. It is practical for interactive and high-quality animation generation.
  arXiv  Detail & Related papers  (2023-11-28T18:54:16Z)
• Decision-Making for Autonomous Vehicles with Interaction-Aware Behavioral Prediction and Social-Attention Neural Network [7.812717451846781]
  We propose a behavioral model that encodes drivers' interacting intentions into latent social-psychological parameters. We develop a receding-horizon optimization-based controller for autonomous vehicle decision-making. We conduct extensive evaluations of the proposed decision-making module, in forced highway merging scenarios.
  arXiv  Detail & Related papers  (2023-10-31T03:31:09Z)
• Tackling Real-World Autonomous Driving using Deep Reinforcement Learning [63.3756530844707]
  In this work, we propose a model-free Deep Reinforcement Learning Planner training a neural network that predicts acceleration and steering angle. In order to deploy the system on board the real self-driving car, we also develop a module represented by a tiny neural network.
  arXiv  Detail & Related papers  (2022-07-05T16:33:20Z)
• Control-Aware Prediction Objectives for Autonomous Driving [78.19515972466063]
  We present control-aware prediction objectives (CAPOs) to evaluate the downstream effect of predictions on control without requiring the planner be differentiable. We propose two types of importance weights that weight the predictive likelihood: one using an attention model between agents, and another based on control variation when exchanging predicted trajectories for ground truth trajectories.
  arXiv  Detail & Related papers  (2022-04-28T07:37:21Z)
• How To Not Drive: Learning Driving Constraints from Demonstration [0.0]
  We propose a new scheme to learn motion planning constraints from human driving trajectories. The behavioral planning is responsible for high-level decision making required to follow traffic rules. The motion planner role is to generate feasible, safe trajectories for a self-driving vehicle to follow.
  arXiv  Detail & Related papers  (2021-10-01T20:47:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.