TeLL-Drive: Enhancing Autonomous Driving with Teacher LLM-Guided Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2502.01387v3
• Date: Thu, 20 Feb 2025 14:09:01 GMT
• Title: TeLL-Drive: Enhancing Autonomous Driving with Teacher LLM-Guided Deep Reinforcement Learning
• Authors: Chengkai Xu, Jiaqi Liu, Shiyu Fang, Yiming Cui, Dong Chen, Peng Hang, Jian Sun,
• Abstract summary: TeLL-Drive is a hybrid framework that integrates a Teacher LLM to guide an attention-based Student DRL policy.<n>A self-attention mechanism then fuses these strategies with the DRL agent's exploration, accelerating policy convergence and boosting robustness.
• Score: 61.33599727106222
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Although Deep Reinforcement Learning (DRL) and Large Language Models (LLMs) each show promise in addressing decision-making challenges in autonomous driving, DRL often suffers from high sample complexity, while LLMs have difficulty ensuring real-time decision making. To address these limitations, we propose TeLL-Drive, a hybrid framework that integrates a Teacher LLM to guide an attention-based Student DRL policy. By incorporating risk metrics, historical scenario retrieval, and domain heuristics into context-rich prompts, the LLM produces high-level driving strategies through chain-of-thought reasoning. A self-attention mechanism then fuses these strategies with the DRL agent's exploration, accelerating policy convergence and boosting robustness across diverse driving conditions. The experimental results, evaluated across multiple traffic scenarios, show that TeLL-Drive outperforms existing baseline methods, including other LLM-based approaches, in terms of success rates, average returns, and real-time feasibility. Ablation studies underscore the importance of each model component, especially the synergy between the attention mechanism and LLM-driven guidance. Finally, we build a virtual-real fusion experimental platform to verify the real-time performance, robustness, and reliability of the algorithm running on real vehicles through vehicle-in-loop experiments.

Related papers

• Large Multimodal Models for Embodied Intelligent Driving: The Next Frontier in Self-Driving? [68.82027978227008]
  This article introduces a novel semantics and policy dual-driven hybrid decision framework to tackle this challenge.<n>The framework merges LMMs for semantic understanding and cognitive representation, and deep reinforcement learning (DRL) for real-time policy optimization.<n>Case study is conducted experimentally to validate the performance superiority of our framework in completing lane-change planning task.
  arXiv  Detail & Related papers  (2026-01-13T11:05:12Z)
• MindDrive: A Vision-Language-Action Model for Autonomous Driving via Online Reinforcement Learning [51.20229133553804]
  Current Vision-Language-Action (VLA) paradigms in autonomous driving primarily rely on Imitation Learning (IL)<n>Online Reinforcement Learning offers a promising pathway to address these issues through trial-and-error learning.<n>We propose MindDrive, a VLA framework comprising a large language model (LLM) with two distinct sets of LoRA parameters.<n>By feeding trajectory-level rewards back into the reasoning space, MindDrive enables trial-and-error learning over a finite set of discrete linguistic driving decisions.
  arXiv  Detail & Related papers  (2025-12-15T18:31:32Z)
• Are LLMs The Way Forward? A Case Study on LLM-Guided Reinforcement Learning for Decentralized Autonomous Driving [9.255259913388096]
  Small, locally deployed Large Language Models (LLMs) can support autonomous highway driving through reward shaping rather than direct control.<n>We present a case study comparing RL-only, LLM-only, and hybrid approaches.<n>Our findings reveal that RL-only agents achieve moderate success rates (73-89%) with reasonable efficiency, LLM-only agents can reach higher success rates (up to 94%) but with severely degraded speed performance, and hybrid approaches consistently fall between these extremes.
  arXiv  Detail & Related papers  (2025-11-16T19:31:42Z)
• AdaDrive: Self-Adaptive Slow-Fast System for Language-Grounded Autonomous Driving [71.55254573283793]
  Existing approaches either activate Large Language Models too frequently, causing excessive computational overhead, or use fixed schedules.<n>We propose AdaDrive, an adaptively collaborative slow-fast framework that optimally determines when and how LLMs contribute to decision-making.<n>AdaDrive provides a flexible, context-aware framework that maximizes decision accuracy without compromising real-time performance.
  arXiv  Detail & Related papers  (2025-11-09T07:05:03Z)
• RL-PLUS: Countering Capability Boundary Collapse of LLMs in Reinforcement Learning with Hybrid-policy Optimization [86.30192066451256]
  We propose RL-PLUS, a novel hybrid-policy optimization approach for Large Language Models (LLMs)<n> RL-PLUS synergizes internal exploitation with external data to achieve stronger reasoning capabilities and surpass the boundaries of base models.<n>We provide both theoretical analysis and extensive experiments to demonstrate the superiority and generalizability of our approach.
  arXiv  Detail & Related papers  (2025-07-31T23:55:29Z)
• ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving [49.07731497951963]
  ReCogDrive is a novel Reinforced Cognitive framework for end-to-end autonomous driving.<n>We introduce a hierarchical data pipeline that mimics the sequential cognitive process of human drivers.<n>We then address the language-action mismatch by injecting the VLM's learned driving priors into a diffusion planner.
  arXiv  Detail & Related papers  (2025-06-09T03:14:04Z)
• HCRMP: A LLM-Hinted Contextual Reinforcement Learning Framework for Autonomous Driving [4.340881027724334]
  Large Language Models (LLMs) with Reinforcement Learning (RL) can enhance autonomous driving (AD) performance in complex scenarios.<n> Evaluations show that state-of-the-art LLM indicates a non-hallucination rate of only approximately 57.95% when assessed on essential driving-related tasks.<n>This paper argues that maintaining relative independence between the LLM and the RL is vital for solving the hallucinations problem.
  arXiv  Detail & Related papers  (2025-05-21T17:47:24Z)
• MLE-Dojo: Interactive Environments for Empowering LLM Agents in Machine Learning Engineering [57.156093929365255]
  Gym-style framework for systematically reinforcement learning, evaluating, and improving autonomous large language model (LLM) agents.<n>MLE-Dojo covers diverse, open-ended MLE tasks carefully curated to reflect realistic engineering scenarios.<n>Its fully executable environment supports comprehensive agent training via both supervised fine-tuning and reinforcement learning.
  arXiv  Detail & Related papers  (2025-05-12T17:35:43Z)
• R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning [87.30285670315334]
  textbfR1-Searcher is a novel two-stage outcome-based RL approach designed to enhance the search capabilities of Large Language Models. Our framework relies exclusively on RL, without requiring process rewards or distillation for a cold start. Our experiments demonstrate that our method significantly outperforms previous strong RAG methods, even when compared to the closed-source GPT-4o-mini.
  arXiv  Detail & Related papers  (2025-03-07T17:14:44Z)
• SenseRAG: Constructing Environmental Knowledge Bases with Proactive Querying for LLM-Based Autonomous Driving [10.041702058108482]
  This study addresses the critical need for enhanced situational awareness in autonomous driving (AD) by leveraging the contextual reasoning capabilities of large language models (LLMs) Unlike traditional perception systems that rely on rigid, label-based annotations, it integrates real-time, multimodal sensor data into a unified, LLMs-readable knowledge base. Experimental results using real-world Vehicle-to-everything (V2X) datasets demonstrate significant improvements in perception and prediction performance.
  arXiv  Detail & Related papers  (2025-01-07T05:15:46Z)
• PKRD-CoT: A Unified Chain-of-thought Prompting for Multi-Modal Large Language Models in Autonomous Driving [8.971981009717284]
  This study explores a seamless integration of MLLMs into autonomous driving systems by proposing a Zero-Shot Chain-of-Thought prompt design named PKRD-CoT.<n>Our design enables MLLMs to tackle problems without prior experience, thereby increasing their utility within unstructured autonomous driving environments.
  arXiv  Detail & Related papers  (2024-12-02T23:08:38Z)
• Robust RL with LLM-Driven Data Synthesis and Policy Adaptation for Autonomous Driving [41.87011820577736]
  This paper introduces RAPID, a novel framework for training mix-of-policy Reinforcement Learning agents. It trains specialized mix-of-policy RL agents using data synthesized by an LLM-based driving agent and online adaptation. It reduces the robustness of LLM knowledge while maintaining adaptability to different tasks.
  arXiv  Detail & Related papers  (2024-10-16T13:43:00Z)
• Large Language Model-Enhanced Reinforcement Learning for Generic Bus Holding Control Strategies [12.599164162404994]
  This study introduces an automatic reward generation paradigm by leveraging the in-context learning and reasoning capabilities of Large Language Models (LLMs) To evaluate the feasibility of the proposed LLM-enhanced RL paradigm, it is applied to various bus holding control scenarios, including a synthetic single-line system and a real-world multi-line system.
  arXiv  Detail & Related papers  (2024-10-14T07:10:16Z)
• From Imitation to Exploration: End-to-end Autonomous Driving based on World Model [24.578178308010912]
  RAMBLE is an end-to-end world model-based RL method for driving decision-making. It can handle complex and dynamic traffic scenarios. It achieves state-of-the-art performance in route completion rate on the CARLA Leaderboard 1.0 and completes all 38 scenarios on the CARLA Leaderboard 2.0.
  arXiv  Detail & Related papers  (2024-10-03T06:45:59Z)
• Towards Interactive and Learnable Cooperative Driving Automation: a Large Language Model-Driven Decision-Making Framework [79.088116316919]
  Connected Autonomous Vehicles (CAVs) have begun to open road testing around the world, but their safety and efficiency performance in complex scenarios is still not satisfactory. This paper proposes CoDrivingLLM, an interactive and learnable LLM-driven cooperative driving framework.
  arXiv  Detail & Related papers  (2024-09-19T14:36:00Z)
• CIMRL: Combining IMitation and Reinforcement Learning for Safe Autonomous Driving [45.05135725542318]
  IMitation and Reinforcement Learning (CIMRL) approach enables training driving policies in simulation through leveraging imitative motion priors and safety constraints. By combining RL and imitation, we demonstrate our method achieves state-of-the-art results in closed loop simulation and real world driving benchmarks.
  arXiv  Detail & Related papers  (2024-06-13T07:31:29Z)
• Probing Multimodal LLMs as World Models for Driving [72.18727651074563]
  We look at the application of Multimodal Large Language Models (MLLMs) in autonomous driving. Despite advances in models like GPT-4o, their performance in complex driving environments remains largely unexplored.
  arXiv  Detail & Related papers  (2024-05-09T17:52:42Z)
• Generalizing Cooperative Eco-driving via Multi-residual Task Learning [6.864745785996583]
  Multi-residual Task Learning (MRTL) is a generic learning framework based on multi-task learning. MRTL decomposes control into nominal components that are effectively solved by conventional control methods and residual terms. We employ MRTL for fleet-level emission reduction in mixed traffic using autonomous vehicles as a means of system control.
  arXiv  Detail & Related papers  (2024-03-07T05:25:34Z)
• Empowering Autonomous Driving with Large Language Models: A Safety Perspective [82.90376711290808]
  This paper explores the integration of Large Language Models (LLMs) into Autonomous Driving systems. LLMs are intelligent decision-makers in behavioral planning, augmented with a safety verifier shield for contextual safety learning. We present two key studies in a simulated environment: an adaptive LLM-conditioned Model Predictive Control (MPC) and an LLM-enabled interactive behavior planning scheme with a state machine.
  arXiv  Detail & Related papers  (2023-11-28T03:13:09Z)
• LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving [87.1164964709168]
  This work employs Large Language Models (LLMs) as a decision-making component for complex autonomous driving scenarios. Extensive experiments demonstrate that our proposed method not only consistently surpasses baseline approaches in single-vehicle tasks, but also helps handle complex driving behaviors even multi-vehicle coordination.
  arXiv  Detail & Related papers  (2023-10-04T17:59:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.