What Matters for Scalable and Robust Learning in End-to-End Driving Planners?

• URL: http://arxiv.org/abs/2603.15185v1
• Date: Mon, 16 Mar 2026 12:20:34 GMT
• Title: What Matters for Scalable and Robust Learning in End-to-End Driving Planners?
• Abstract summary: We re-examine the impact of architectural patterns on closed-loop performance.<n>We introduce BevAD, a novel lightweight and highly scalable end-to-end driving architecture.
• Score: 45.17722693412255
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: End-to-end autonomous driving has gained significant attention for its potential to learn robust behavior in interactive scenarios and scale with data. Popular architectures often build on separate modules for perception and planning connected through latent representations, such as bird's eye view feature grids, to maintain end-to-end differentiability. This paradigm emerged mostly on open-loop datasets, with evaluation focusing not only on driving performance, but also intermediate perception tasks. Unfortunately, architectural advances that excel in open-loop often fail to translate to scalable learning of robust closed-loop driving. In this paper, we systematically re-examine the impact of common architectural patterns on closed-loop performance: (1) high-resolution perceptual representations, (2) disentangled trajectory representations, and (3) generative planning. Crucially, our analysis evaluates the combined impact of these patterns, revealing both unexpected limitations as well as underexplored synergies. Building on these insights, we introduce BevAD, a novel lightweight and highly scalable end-to-end driving architecture. BevAD achieves 72.7% success rate on the Bench2Drive benchmark and demonstrates strong data-scaling behavior using pure imitation learning. Our code and models are publicly available here: https://dmholtz.github.io/bevad/

Related papers

• PRISM: Privileged Probabilistic Latent Supervision for End-to-End Autonomous Driving Motion Planning [18.741741219314836]
  End-to-end autonomous driving systems integrate perception, prediction, and planning into a single differentiable architecture.<n>Recent works have integrated vision-language model (VLM) supervision for latent features to address this, yielding substantial empirical gains.<n>Our investigation into this methodology reveals that the performance gains stem not from VLM reasoning capabilities, as previously assumed, but rather from the latent connections forged between the E2E AD model and ground-truth (GT) data during training.
  arXiv  Detail & Related papers  (2026-08-02T12:33:52Z)
• DriveReward: A Comprehensive Dataset and Generative Vision-Language Reward Model for Autonomous Driving [17.749010844640388]
  Reward models play a pivotal role in reinforcement learning (RL) and multi-modal trajectory selection for autonomous driving.<n>While Vision-Language Models (VLMs) have demonstrated feasibility as reward models in other domains, their effectiveness in driving tasks remains underexplored.<n>We bridge this gap by introducing DriveReward, a reasoning trajectory evaluation dataset rigorously labeled via temporally-grounded visual guidance.
  arXiv  Detail & Related papers  (2026-06-07T09:05:49Z)
• BikeActions: An Open Platform and Benchmark for Cyclist-Centric VRU Action Recognition [0.2339805471804333]
  FUSE-Bike is the first fully open perception platform of its kind.<n>BikeActions is a novel multi-modal dataset comprising 852 annotated samples across 5 distinct action classes.<n>We establish a rigorous benchmark by evaluating state-of-the-art graph convolution and transformer-based models on our publicly released data splits.
  arXiv  Detail & Related papers  (2026-01-15T15:47:46Z)
• LEAD: Minimizing Learner-Expert Asymmetry in End-to-End Driving [29.34610793694948]
  We study how misalignment between privileged expert demonstrations and sensor-based student observations can limit the effectiveness of imitation learning.<n>Our TransFuser v6 (TFv6) student policy achieves a new state of the art on all major publicly available CARLA closed-loop benchmarks.
  arXiv  Detail & Related papers  (2025-12-23T18:07:43Z)
• Data Scaling Laws for End-to-End Autonomous Driving [83.85463296830743]
  We evaluate the performance of a simple end-to-end driving architecture on internal driving datasets ranging in size from 16 to 8192 hours.<n>Specifically, we investigate how much additional training data is needed to achieve a target performance gain.
  arXiv  Detail & Related papers  (2025-04-06T03:23:48Z)
• DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Driving [62.62464518137153]
  DriveTransformer is a simplified E2E-AD framework for the ease of scaling up.<n>It is composed of three unified operations: task self-attention, sensor cross-attention, temporal cross-attention.<n>It achieves state-of-the-art performance in both simulated closed-loop benchmark Bench2Drive and real world open-loop benchmark nuScenes with high FPS.
  arXiv  Detail & Related papers  (2025-03-07T11:41:18Z)
• Enhancing End-to-End Autonomous Driving with Latent World Model [78.22157677787239]
  We propose a novel self-supervised learning approach using the LAtent World model (LAW) for end-to-end driving.<n> LAW predicts future scene features based on current features and ego trajectories.<n>This self-supervised task can be seamlessly integrated into perception-free and perception-based frameworks.
  arXiv  Detail & Related papers  (2024-06-12T17:59:21Z)
• Deep Active Inference for Pixel-Based Discrete Control: Evaluation on the Car Racing Problem [0.0]
  We study the performance of a deep active inference (dAIF) agent on OpenAI's car racing benchmark. State inference and control are learned end-to-end by optimizing the expected free energy. vanilla dAIF does not reach state-of-the-art performance compared to other world model approaches.
  arXiv  Detail & Related papers  (2021-09-09T10:33:36Z)
• Fine-Grained Vehicle Perception via 3D Part-Guided Visual Data Augmentation [77.60050239225086]
  We propose an effective training data generation process by fitting a 3D car model with dynamic parts to vehicles in real images. Our approach is fully automatic without any human interaction. We present a multi-task network for VUS parsing and a multi-stream network for VHI parsing.
  arXiv  Detail & Related papers  (2020-12-15T03:03:38Z)
• Spatiotemporal Relationship Reasoning for Pedestrian Intent Prediction [57.56466850377598]
  Reasoning over visual data is a desirable capability for robotics and vision-based applications. In this paper, we present a framework on graph to uncover relationships in different objects in the scene for reasoning about pedestrian intent. Pedestrian intent, defined as the future action of crossing or not-crossing the street, is a very crucial piece of information for autonomous vehicles.
  arXiv  Detail & Related papers  (2020-02-20T18:50:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.