Do LLM Modules Generalize? A Study on Motion Generation for Autonomous Driving

• URL: http://arxiv.org/abs/2509.02754v1
• Date: Tue, 02 Sep 2025 19:02:49 GMT
• Title: Do LLM Modules Generalize? A Study on Motion Generation for Autonomous Driving
• Authors: Mingyi Wang, Jingke Wang, Tengju Ye, Junbo Chen, Kaicheng Yu,
• Abstract summary: We present a comprehensive evaluation of five key LLM modules.<n>We demonstrate that, when appropriately adapted, these modules can significantly improve performance for autonomous driving motion generation.<n>In addition, we identify which techniques can be effectively transferred, analyze the potential reasons for the failure of others, and discuss the specific adaptations needed for autonomous driving scenarios.
• Score: 15.903491909277745
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent breakthroughs in large language models (LLMs) have not only advanced natural language processing but also inspired their application in domains with structurally similar problems--most notably, autonomous driving motion generation. Both domains involve autoregressive sequence modeling, token-based representations, and context-aware decision making, making the transfer of LLM components a natural and increasingly common practice. However, despite promising early attempts, a systematic understanding of which LLM modules are truly transferable remains lacking. In this paper, we present a comprehensive evaluation of five key LLM modules--tokenizer design, positional embedding, pre-training paradigms, post-training strategies, and test-time computation--within the context of motion generation for autonomous driving. Through extensive experiments on the Waymo Sim Agents benchmark, we demonstrate that, when appropriately adapted, these modules can significantly improve performance for autonomous driving motion generation. In addition, we identify which techniques can be effectively transferred, analyze the potential reasons for the failure of others, and discuss the specific adaptations needed for autonomous driving scenarios. We evaluate our method on the Sim Agents task and achieve competitive results.

Related papers

• Software Defined Vehicle Code Generation: A Few-Shot Prompting Approach [0.0]
  General-purpose large language models (LLMs) have demonstrated transformative potential across domains.<n>This study proposes using prompts, a common and basic strategy to interact with LLMs and redirect their responses.<n>Using only system prompts with an appropriate and efficient prompt structure designed using advanced prompt engineering techniques, LLMs can be crafted without requiring a training session or access to their base design.
  arXiv  Detail & Related papers  (2025-11-06T22:27:39Z)
• Large Multimodal Models-Empowered Task-Oriented Autonomous Communications: Design Methodology and Implementation Challenges [31.57528074626831]
  Large language models (LLMs) and large multimodal models (LMMs) have achieved unprecedented breakthrough.<n>This article focuses on task-oriented autonomous communications with LLMs/LMMs.<n>We show that the proposed LLM/LMM-aided autonomous systems significantly outperform conventional and discriminative deep learning (DL) model-based techniques.
  arXiv  Detail & Related papers  (2025-10-23T15:08:58Z)
• Discrete Diffusion for Reflective Vision-Language-Action Models in Autonomous Driving [55.13109926181247]
  We introduce ReflectDrive, a learning-based framework that integrates a reflection mechanism for safe trajectory generation via discrete diffusion.<n>Central to our approach is a safety-aware reflection mechanism that performs iterative self-correction without gradient.<n>Our method begins with goal-conditioned trajectory generation to model multi-modal driving behaviors.
  arXiv  Detail & Related papers  (2025-09-24T13:35:15Z)
• Self-Improving Model Steering [13.424901485601994]
  We present SIMS, the first self-improving model-steering framework that operates without relying on external supervision.<n>At its core, SIMS autonomously generates and refines contrastive samples through iterative self-improvement cycles.<n>We show that SIMS substantially outperforms existing methods in steering effectiveness and adaptability.
  arXiv  Detail & Related papers  (2025-07-11T18:52:32Z)
• ADRD: LLM-Driven Autonomous Driving Based on Rule-based Decision Systems [6.950569256149267]
  We introduce the ADRD framework, which integrates three core modules: the Information Module, the Agents Module, and the Testing Module.<n>The framework exhibits superior performance in autonomous driving decision tasks.<n>This is the first work that integrates large language models with rule-based systems for autonomous driving decision-making.
  arXiv  Detail & Related papers  (2025-06-17T08:18:20Z)
• LightEMMA: Lightweight End-to-End Multimodal Model for Autonomous Driving [9.447298958886265]
  Vision-Language Models (VLMs) have demonstrated significant potential for end-to-end autonomous driving.<n>We introduce LightEMMA, a Lightweight End-to-End Multimodal Model for Autonomous driving.<n>We construct twelve autonomous driving agents using various VLMs and evaluate their performance on the nuScenes prediction task.
  arXiv  Detail & Related papers  (2025-05-01T04:12:41Z)
• Scaling Autonomous Agents via Automatic Reward Modeling And Planning [52.39395405893965]
  Large language models (LLMs) have demonstrated remarkable capabilities across a range of tasks.<n>However, they still struggle with problems requiring multi-step decision-making and environmental feedback.<n>We propose a framework that can automatically learn a reward model from the environment without human annotations.
  arXiv  Detail & Related papers  (2025-02-17T18:49:25Z)
• Large Language Models for Autonomous Driving (LLM4AD): Concept, Benchmark, Experiments, and Challenges [15.52530518623987]
  Large Language Models (LLMs) have the potential to enhance various aspects of autonomous driving systems.<n>In this paper, we first introduce the novel concept of designing LLMs for autonomous driving (LLM4AD)<n>We conduct a series of experiments on real-world vehicle platforms, thoroughly evaluating the performance and potential of our LLM4AD systems.
  arXiv  Detail & Related papers  (2024-10-20T04:36:19Z)
• Probing Multimodal LLMs as World Models for Driving [72.18727651074563]
  We look at the application of Multimodal Large Language Models (MLLMs) in autonomous driving. Despite advances in models like GPT-4o, their performance in complex driving environments remains largely unexplored.
  arXiv  Detail & Related papers  (2024-05-09T17:52:42Z)
• LLM4Drive: A Survey of Large Language Models for Autonomous Driving [62.10344445241105]
  Large language models (LLMs) have demonstrated abilities including understanding context, logical reasoning, and generating answers. In this paper, we systematically review a research line about textitLarge Language Models for Autonomous Driving (LLM4AD).
  arXiv  Detail & Related papers  (2023-11-02T07:23:33Z)
• LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving [84.31119464141631]
  This work employs Large Language Models (LLMs) as a decision-making component for complex autonomous driving scenarios.<n>Extensive experiments demonstrate that our proposed method not only consistently surpasses baseline approaches in single-vehicle tasks, but also helps handle complex driving behaviors even multi-vehicle coordination.
  arXiv  Detail & Related papers  (2023-10-04T17:59:49Z)
• SELF: Self-Evolution with Language Feedback [68.6673019284853]
  'SELF' (Self-Evolution with Language Feedback) is a novel approach to advance large language models. It enables LLMs to self-improve through self-reflection, akin to human learning processes. Our experiments in mathematics and general tasks demonstrate that SELF can enhance the capabilities of LLMs without human intervention.
  arXiv  Detail & Related papers  (2023-10-01T00:52:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.