Zero-Shot Cross-City Generalization in End-to-End Autonomous Driving: Self-Supervised versus Supervised Representations

• URL: http://arxiv.org/abs/2603.11417v1
• Date: Thu, 12 Mar 2026 01:19:32 GMT
• Title: Zero-Shot Cross-City Generalization in End-to-End Autonomous Driving: Self-Supervised versus Supervised Representations
• Abstract summary: We investigate zero-shot cross-city generalization in end-to-end trajectory planning.<n>Self-supervised visual representations improve transfer across cities.<n>These results establish zero-shot geographic transfer as a necessary test for evaluating end-to-end autonomous driving systems.
• Score: 9.18632648031395
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: End-to-end autonomous driving models are typically trained on multi-city datasets using supervised ImageNet-pretrained backbones, yet their ability to generalize to unseen cities remains largely unexamined. When training and evaluation data are geographically mixed, models may implicitly rely on city-specific cues, masking failure modes that would occur under real domain shifts when generalizing to new locations. In this work we investigate zero-shot cross-city generalization in end-to-end trajectory planning and ask whether self-supervised visual representations improve transfer across cities. We conduct a comprehensive study by integrating self-supervised backbones (I-JEPA, DINOv2, and MAE) into planning frameworks. We evaluate performance under strict geographic splits on nuScenes in the open-loop setting and on NAVSIM in the closed-loop evaluation protocol. Our experiments reveal a substantial generalization gap when transferring models relying on traditional supervised backbones across cities with different road topologies and driving conventions, particularly when transferring from right-side to left-side driving environments. Self-supervised representation learning reduces this gap. In open-loop evaluation, a supervised backbone exhibits severe inflation when transferring from Boston to Singapore (L2 displacement ratio 9.77x, collision ratio 19.43x), whereas domain-specific self-supervised pretraining reduces this to 1.20x and 0.75x respectively. In closed-loop evaluation, self-supervised pretraining improves PDMS by up to 4 percent for all single-city training cities. These results show that representation learning strongly influences the robustness of cross-city planning and establish zero-shot geographic transfer as a necessary test for evaluating end-to-end autonomous driving systems.

Related papers

• Inferring Urban Mobility Interactions from Aggregated Dynamics [45.73534370352232]
  Real-time urban governance depends not only on knowing where people are, but on how they move between places.<n>Here we show that this directional structure need not be observed to be known.<n>We infer future OD flows from area-level counts alone across twelve mobility datasets from cities in the United States and China.
  arXiv  Detail & Related papers  (2026-09-07T11:13:51Z)
• Shift & Drift: A Zero-Shot Benchmark for Generalizable and Robust Autonomous Driving Motion Planning [60.888463449435484]
  We present Shift & Drift, a novel dual-track benchmark designed to rigorously stress-test motion planners across two critical axes of distribution shift.<n>We show that imitation learning methods achieve high scores in ID benchmarks, but suffer from persistent drift when subjected to temporally correlated actuation noise.<n>Our findings reveal an empirical trade-off between imitation fidelity and closed-loop resilience, providing the community with a rigorous benchmark to evaluate progress toward reliable deployment.
  arXiv  Detail & Related papers  (2026-07-08T18:23:48Z)
• Learning Direct Control Policies with Flow Matching for Autonomous Driving [32.505127447635864]
  Flow-matching planner for autonomous driving generates actionable trajectories defined by acceleration and curvature.<n>Model is conditioned on a bird's-eye-view (BEV) control profiles of the surrounding scene.<n>We train exclusively on urban scenarios collected from a 2D traffic simulator with reactive agents.
  arXiv  Detail & Related papers  (2026-05-14T13:37:48Z)
• MAPLE: Latent Multi-Agent Play for End-to-End Autonomous Driving [62.43744546817599]
  Vision-language--action (VLA) models are effective as end-to-end motion planners, but can be brittle when evaluated in closed-loop settings.<n>We propose MAPLE, a novel framework for reactive, multi-agent rollout of a dynamic driving scenario in the latent space of the VLA model.<n> MAPLE achieves state-of-the-art driving performance on Bench2Drive and demonstrates scalable, closed-loop multi-agent play for robust E2E autonomous driving systems.
  arXiv  Detail & Related papers  (2026-05-13T23:35:14Z)
• Comparative Analysis of Polygon-Based and Global Machine Learning Models for Bus Occupancy Prediction [0.0]
  This paper introduces a novel framework that enhances bus ridership prediction by integrating a spatial clustering methodology with multi-dimensional feature analysis.<n>By clustering the urban area into distinct regions, based on the principle that bus stops in close proximity share similar ridership characteristics, a separate local forecasting model is trained for each of these clusters.<n>The findings suggest that a spatially-aware, localized modeling strategy is effective for public transport prediction, paving the way for more targeted and efficient service improvements.
  arXiv  Detail & Related papers  (2026-04-30T15:35:23Z)
• Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail [85.47497935739936]
  Alpamayo-R1 (AR1) is a vision-language-action model that integrates Chain of Causation reasoning with trajectory planning.<n>We show AR1 achieves 12% improvement in planning accuracy on challenging cases compared to a trajectory-only baseline.<n>We plan to release AR1 models and a subset of the CoC in a future update.
  arXiv  Detail & Related papers  (2025-10-30T01:25:34Z)
• Realistic Urban Traffic Generator using Decentralized Federated Learning for the SUMO simulator [2.281163408378731]
  This work introduces DesRUTGe, a novel framework that integrates Deep Reinforcement Learning agents with the SUMO simulator to generate realistic 24-hour traffic patterns.<n>A key innovation of DesRUTGe is its use of Decentralized Federated Learning (DFL), wherein each traffic detector and its corresponding urban zone function as an independent learning node.
  arXiv  Detail & Related papers  (2025-06-09T17:51:45Z)
• Mitigating Covariate Shift in Imitation Learning for Autonomous Vehicles Using Latent Space Generative World Models [60.87795376541144]
  A world model is a neural network capable of predicting an agent's next state given past states and actions.<n>During end-to-end training, our policy learns how to recover from errors by aligning with states observed in human demonstrations.<n>We present qualitative and quantitative results, demonstrating significant improvements upon prior state of the art in closed-loop testing.
  arXiv  Detail & Related papers  (2024-09-25T06:48:25Z)
• Planning with Adaptive World Models for Autonomous Driving [50.4439896514353]
  We present nuPlan, a real-world motion planning benchmark that captures multi-agent interactions.<n>We learn to model such unique behaviors with BehaviorNet, a graph convolutional neural network (GCNN)<n>We also present AdaptiveDriver, a model-predictive control (MPC) based planner that unrolls different world models conditioned on BehaviorNet's predictions.
  arXiv  Detail & Related papers  (2024-06-15T18:53:45Z)
• Arrival Time Prediction for Autonomous Shuttle Services in the Real World: Evidence from Five Cities [3.6294895527930504]
  This study presents an AT prediction system for autonomous shuttles, utilizing separate models for dwell and running time predictions. To accurately handle the case of a shuttle bypassing a stop, we propose a hierarchical model combining a random forest classifier and a GNN. The results for the final AT prediction are promising, showing low errors even when predicting several stops ahead.
  arXiv  Detail & Related papers  (2024-01-10T18:41:39Z)
• Learning to Drive Anywhere [38.547150940396904]
  We propose AnyD, a single geographically-aware conditional imitation learning model. Our key insight is to introduce a high-capacity geo-location-based channel attention mechanism. Our proposed approach can efficiently scale across inherently imbalanced data distributions and location-dependent events.
  arXiv  Detail & Related papers  (2023-09-21T17:55:36Z)
• Hierarchical Model-Based Imitation Learning for Planning in Autonomous Driving [47.59287162318435]
  We demonstrate the first large-scale application of model-based generative adversarial imitation learning (MGAIL) to the task of dense urban self-driving. We augment standard MGAIL using a hierarchical model to enable generalization to arbitrary goal routes, and measure performance using a closed-loop evaluation framework with simulated interactive agents. We train policies from expert trajectories collected from real vehicles driving over 100,000 miles in San Francisco, and demonstrate a steerable policy that can navigate robustly even in a zero-shot setting.
  arXiv  Detail & Related papers  (2022-10-18T02:15:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.