LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoning
- URL: http://arxiv.org/abs/2401.00125v1
- Date: Sat, 30 Dec 2023 02:53:45 GMT
- Title: LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoning
- Authors: S P Sharan, Francesco Pittaluga, Vijay Kumar B G, Manmohan Chandraker
- Abstract summary: We develop a novel hybrid planner that leverages a conventional rule-based planner in conjunction with an LLM-based planner.
Our approach navigates complex scenarios which existing planners struggle with, produces well-reasoned outputs while also remaining grounded through working alongside the rule-based approach.
- Score: 65.86754998249224
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Although planning is a crucial component of the autonomous driving stack,
researchers have yet to develop robust planning algorithms that are capable of
safely handling the diverse range of possible driving scenarios. Learning-based
planners suffer from overfitting and poor long-tail performance. On the other
hand, rule-based planners generalize well, but might fail to handle scenarios
that require complex driving maneuvers. To address these limitations, we
investigate the possibility of leveraging the common-sense reasoning
capabilities of Large Language Models (LLMs) such as GPT4 and Llama2 to
generate plans for self-driving vehicles. In particular, we develop a novel
hybrid planner that leverages a conventional rule-based planner in conjunction
with an LLM-based planner. Guided by commonsense reasoning abilities of LLMs,
our approach navigates complex scenarios which existing planners struggle with,
produces well-reasoned outputs while also remaining grounded through working
alongside the rule-based approach. Through extensive evaluation on the nuPlan
benchmark, we achieve state-of-the-art performance, outperforming all existing
pure learning- and rule-based methods across most metrics. Our code will be
available at https://llmassist.github.io.
Related papers
- Asynchronous Large Language Model Enhanced Planner for Autonomous Driving [26.72215912937613]
AsyncDriver is a new framework designed to guide real-time planners in making precise and controllable trajectory predictions.
By capitalizing on the asynchronous nature of their inference frequencies, our approach have successfully reduced the computational cost introduced by LLM.
Experiments show that our approach achieves superior closed-loop evaluation performance on nuPlan's challenging scenarios.
arXiv Detail & Related papers (2024-06-20T17:59:03Z) - Exploring and Benchmarking the Planning Capabilities of Large Language Models [57.23454975238014]
This work lays the foundations for improving planning capabilities of large language models (LLMs)
We construct a comprehensive benchmark suite encompassing both classical planning benchmarks and natural language scenarios.
We investigate the use of many-shot in-context learning to enhance LLM planning, exploring the relationship between increased context length and improved planning performance.
arXiv Detail & Related papers (2024-06-18T22:57:06Z) - Can Vehicle Motion Planning Generalize to Realistic Long-tail Scenarios? [11.917542484123134]
Real-world autonomous driving systems must make safe decisions in the face of rare and diverse traffic scenarios.
Current state-of-the-art planners are mostly evaluated on real-world datasets like nuScenes (open-loop) or nuPlan (closed-loop)
arXiv Detail & Related papers (2024-04-11T08:57:48Z) - Empowering Autonomous Driving with Large Language Models: A Safety Perspective [82.90376711290808]
This paper explores the integration of Large Language Models (LLMs) into Autonomous Driving systems.
LLMs are intelligent decision-makers in behavioral planning, augmented with a safety verifier shield for contextual safety learning.
We present two key studies in a simulated environment: an adaptive LLM-conditioned Model Predictive Control (MPC) and an LLM-enabled interactive behavior planning scheme with a state machine.
arXiv Detail & Related papers (2023-11-28T03:13:09Z) - LanguageMPC: Large Language Models as Decision Makers for Autonomous
Driving [87.1164964709168]
This work employs Large Language Models (LLMs) as a decision-making component for complex autonomous driving scenarios.
Extensive experiments demonstrate that our proposed method not only consistently surpasses baseline approaches in single-vehicle tasks, but also helps handle complex driving behaviors even multi-vehicle coordination.
arXiv Detail & Related papers (2023-10-04T17:59:49Z) - GPT-Driver: Learning to Drive with GPT [47.14350537515685]
We present a simple yet effective approach that can transform the OpenAI GPT-3.5 model into a reliable motion planner for autonomous vehicles.
We capitalize on the strong reasoning capabilities and generalization potential inherent to Large Language Models (LLMs)
We evaluate our approach on the large-scale nuScenes dataset, and extensive experiments substantiate the effectiveness, generalization ability, and interpretability of our GPT-based motion planner.
arXiv Detail & Related papers (2023-10-02T17:59:57Z) - AdaPlanner: Adaptive Planning from Feedback with Language Models [56.367020818139665]
Large language models (LLMs) have recently demonstrated the potential in acting as autonomous agents for sequential decision-making tasks.
We propose a closed-loop approach, AdaPlanner, which allows the LLM agent to refine its self-generated plan adaptively in response to environmental feedback.
To mitigate hallucination, we develop a code-style LLM prompt structure that facilitates plan generation across a variety of tasks, environments, and agent capabilities.
arXiv Detail & Related papers (2023-05-26T05:52:27Z) - PlanBench: An Extensible Benchmark for Evaluating Large Language Models
on Planning and Reasoning about Change [34.93870615625937]
PlanBench is a benchmark suite based on the kinds of domains used in the automated planning community.
PlanBench provides sufficient diversity in both the task domains and the specific planning capabilities.
arXiv Detail & Related papers (2022-06-21T16:15:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.