DrivingAgent: Design and Scheduling Agents for Autonomous Driving Systems

• URL: http://arxiv.org/abs/2606.12236v2
• Date: Thu, 11 Jun 2026 01:12:41 GMT
• Title: DrivingAgent: Design and Scheduling Agents for Autonomous Driving Systems
• Abstract summary: We propose DrivingAgent, a novel agent framework tailored to the dual challenges of autonomous driving system design and scheduling.<n>In the design phase, DrivingAgent automates module development by interpreting system architecture, generating code, and validating modules via super-network training.<n>In the scheduling phase, it employs a lightweight LLM trained with reinforcement learning to dynamically orchestrate system modules in real time, supported by a structured memory that integrates long-term storage with timestamped short-term context.
• Score: 44.93698347738791
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Many autonomous driving systems are increasingly incorporating foundation models to improve generalization and handle long-tail scenarios. However, this trend introduces two key challenges: (i) the manual and labor-intensive process of designing and integrating new models, and (ii) the lack of intelligent, dynamic scheduling mechanisms to meet strict real-time constraints. While Large Language Model (LLM)-based agents offer a promising avenue for automation, existing frameworks are ill-suited for autonomous driving. Specifically, they fail to distinguish between the fundamentally different requirements of system design and real-time scheduling, treat modules as opaque black boxes, and are not designed for continuous operation. To address these limitations, we propose DrivingAgent, a novel agent framework tailored to the dual challenges of autonomous driving system design and scheduling. In the design phase, DrivingAgent automates module development by interpreting system architecture, generating code, and validating modules via super-network training. In the scheduling phase, it employs a lightweight LLM trained with reinforcement learning to dynamically orchestrate system modules in real time, supported by a structured memory that integrates long-term storage with timestamped short-term context. Experimental results demonstrate that DrivingAgent achieves a superior speed--accuracy trade-off on both the nuScenes and Bench2Drive benchmarks.

Related papers

• Dual Process Motion Planning [53.570848178455776]
  We introduce a dual-process architecture that combines the strengths of robust reasoning and learning.<n>Our framework integrates state-of-the-art symbolic solvers as a System-2'' component with experience-driven System-1'' modules.
  arXiv  Detail & Related papers  (2026-09-01T13:52:12Z)
• Model-Driven Discipline for Multi-Agent LLMs: Requirement-to-Verification Generation of Traceable System Models [62.62160094849002]
  RADIANT is an engineering methodology that combines MDE with Multi-Agent Large Language Models.<n>It automatically generates heterogeneous models across engineering phases.<n>It provides exact, automated change-impact analysis.
  arXiv  Detail & Related papers  (2026-07-18T08:50:55Z)
• OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models [69.2503510410147]
  We present a unified autonomous driving framework built upon a pretrained VLM.<n>We demonstrate that pretrained VLM attention exhibits strong transferability beyond pure language modeling.<n>Experiments on end-to-end autonomous driving benchmarks demonstrate state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-04-20T07:50:00Z)
• Bridging Large-Model Reasoning and Real-Time Control via Agentic Fast-Slow Planning [42.15812524999639]
  Agentic Fast-Slow Planning is a hierarchical framework that decouples perception, reasoning, planning, and control across natural timescales.<n>We show that Agentic Fast-Slow Planning improves robustness under perturbations, reducing lateral deviation by up to 45% and completion time by over 12% compared to pure MPC and an A*-guided MPC baseline.
  arXiv  Detail & Related papers  (2026-04-02T06:34:29Z)
• Loosely-Structured Software: Engineering Context, Structure, and Evolution Entropy in Runtime-Rewired Multi-Agent Systems [9.34019956999278]
  Building robust multi-agent systems requires engineering discipline focused on architecture to manage complexity under uncertainty.<n>This paper introduces emphLoosely-Structured Software (LSS), a new class of software systems that shifts the engineering focus from constructing deterministic logic to managing the runtime entropy.
  arXiv  Detail & Related papers  (2026-03-16T02:07:34Z)
• AROLA: A Modular Layered Architecture for Scaled Autonomous Racing [0.6372261626436676]
  In this work, AROLA is introduced as a modular, layered software architecture in which fragmented and monolithic designs are reorganized into interchangeable layers and components connected through standardized ROS 2 interfaces.<n>The autonomous-driving pipeline is decomposed into sensing, pre-processing, perception, localization and mapping, planning, behavior, control, and actuation.<n>A Race Monitor framework is introduced as a lightweight system through which lap timing, trajectory quality, and computational load are logged in real time and standardized post-race analyses are generated.
  arXiv  Detail & Related papers  (2026-02-02T19:46:34Z)
• A Lightweight Modular Framework for Constructing Autonomous Agents Driven by Large Language Models: Design, Implementation, and Applications in AgentForge [1.932555230783329]
  Lightweight, open-source Python framework designed to democratize the construction of LLM-driven autonomous agents.<n>AgentForge introduces three key innovations: (1) a composable skill abstraction that enables fine-grained task decomposition with formally defined input-output contracts, (2) a unified backend interface supporting seamless switching between cloud-based APIs and local inference engines, and (3) a declarative YAML-based configuration system that separates agent logic from implementation details.
  arXiv  Detail & Related papers  (2026-01-19T20:33:26Z)
• SGDrive: Scene-to-Goal Hierarchical World Cognition for Autonomous Driving [52.02379432801349]
  We propose SGDrive, a novel framework that structures the VLM's representation learning around driving-specific knowledge hierarchies.<n>Built upon a pre-trained VLM backbone, SGDrive decomposes driving understanding into a scene-agent-goal hierarchy that mirrors human driving cognition.
  arXiv  Detail & Related papers  (2026-01-09T08:55:42Z)
• AdaDrive: Self-Adaptive Slow-Fast System for Language-Grounded Autonomous Driving [71.55254573283793]
  Existing approaches either activate Large Language Models too frequently, causing excessive computational overhead, or use fixed schedules.<n>We propose AdaDrive, an adaptively collaborative slow-fast framework that optimally determines when and how LLMs contribute to decision-making.<n>AdaDrive provides a flexible, context-aware framework that maximizes decision accuracy without compromising real-time performance.
  arXiv  Detail & Related papers  (2025-11-09T07:05:03Z)
• Pangu Embedded: An Efficient Dual-system LLM Reasoner with Metacognition [95.54406667705999]
  Pangu Embedded is an efficient Large Language Model (LLM) reasoner developed on Ascend Neural Processing Units (NPUs)<n>It addresses the significant computational costs and inference latency challenges prevalent in existing reasoning-optimized LLMs.<n>It delivers rapid responses and state-of-the-art reasoning quality within a single, unified model architecture.
  arXiv  Detail & Related papers  (2025-05-28T14:03:02Z)
• FASIONAD : FAst and Slow FusION Thinking Systems for Human-Like Autonomous Driving with Adaptive Feedback [15.805379735361862]
  This paper presents FASIONAD, a novel dual-system framework inspired by the cognitive model "Thinking, Fast and Slow"<n>The fast system handles routine navigation tasks using rapid, data-driven path planning, while the slow system focuses on complex reasoning and decision-making in challenging or unfamiliar situations.<n>Visual prompts generated by the fast system enable human-like reasoning in the slow system, which provides high-quality feedback to enhance the fast system's decision-making.
  arXiv  Detail & Related papers  (2024-11-27T03:14:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.