HEAT: Heterogeneous End-to-End Autonomous Driving via Trajectory-Guided World Models

• URL: http://arxiv.org/abs/2605.19631v1
• Date: Tue, 19 May 2026 10:12:01 GMT
• Title: HEAT: Heterogeneous End-to-End Autonomous Driving via Trajectory-Guided World Models
• Abstract summary: End-to-end autonomous driving has emerged as a compelling alternative to traditional modular pipelines.<n>We propose a trajectory-driven learning paradigm that organizes training around planning trajectories.<n>We show that a single unified model can be trained on heterogeneous datasets while maintaining strong performance within each domain.
• Score: 50.18051979020699
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: End-to-end autonomous driving has emerged as a compelling alternative to traditional modular pipelines by directly mapping raw sensor data to driving actions. While recent approaches achieve strong performance on single-domain datasets, their performance degrades significantly when trained jointly across multiple heterogeneous domains. In practice, however, autonomous systems must operate across diverse environments with heterogeneous distributions, including different cities, sensor configurations, and traffic patterns, without domain-specific retraining. This gap highlights a key challenge in multi-domain learning: domain-specific variations across heterogeneous domains introduce conflicting learning signals, driving models toward compromised solutions that are suboptimal across domains. To address this, we propose a trajectory-driven learning paradigm that organizes training around planning trajectories, enabling the model to capture domain-invariant representations of driving intent. Furthermore, we incorporate a world model that predicts future latent features conditioned on ego actions, improving feature consistency and mitigating domain-induced biases. We evaluate our approach on three benchmarks, nuScenes, NAVSIM, and the Waymo end-to-end dataset, and show substantial improvements over existing methods across all domains. Our results demonstrate that a single unified model can be trained on heterogeneous datasets while maintaining strong performance within each domain, highlighting a step toward scalable real-world deployment. We will make our code publicly available.

Related papers

• An Adversarial Zero-Shot Learning Approach for Anomaly Detection in Multivariate IoT Traffic Data [7.858819231575403]
  Anomaly detection in Internet of Things networks presents unique challenges due to diversity of devices, lack of labeled data, and domain variability across environments.<n>We propose a novel framework for multivariate time-series anomaly detection that leverages adversarial learning and contrastive loss within a sequence-based Variational Autoencoder architecture.
  arXiv  Detail & Related papers  (2026-09-03T08:07:06Z)
• DOME: Learning Transferable Domain Variables from Sparse Supervision for Test-Time Adaptation [61.70497741321102]
  Test-time adaptation aims to align a model to shifting test domains using only unlabeled streaming data.<n>Most existing methods implicitly infer a single global domain distribution, ignoring the multidimensional and sample-specific nature of real-world domain shifts.<n>We propose DOME, an effective domain encoder that explicitly models each sample's domain in a zero-shot manner.
  arXiv  Detail & Related papers  (2026-06-02T03:24:03Z)
• Let Synthetic Data Shine: Domain Reassembly and Soft-Fusion for Single Domain Generalization [68.41367635546183]
  Single Domain Generalization aims to train models with consistent performance across diverse scenarios using data from a single source.<n>We propose Discriminative Domain Reassembly and Soft-Fusion (DRSF), a training framework leveraging synthetic data to improve model generalization.
  arXiv  Detail & Related papers  (2025-03-17T18:08:03Z)
• Multisource Collaborative Domain Generalization for Cross-Scene Remote Sensing Image Classification [57.945437355714155]
  Cross-scene image classification aims to transfer prior knowledge of ground materials to annotate regions with different distributions.<n>Existing approaches focus on single-source domain generalization to unseen target domains.<n>We propose a novel multi-source collaborative domain generalization framework (MS-CDG) based on homogeneity and heterogeneity characteristics of multi-source remote sensing data.
  arXiv  Detail & Related papers  (2024-12-05T06:15:08Z)
• xTED: Cross-Domain Adaptation via Diffusion-Based Trajectory Editing [21.37585797507323]
  Cross-domain policy transfer methods mostly aim at learning domain correspondences or corrections to facilitate policy learning.<n>We propose the Cross-Domain Trajectory EDiting framework that employs a specially designed diffusion model for cross-domain trajectory adaptation.<n>Our proposed model architecture effectively captures the intricate dependencies among states, actions, and rewards, as well as the dynamics patterns within target data.
  arXiv  Detail & Related papers  (2024-09-13T10:07:28Z)
• Towards Full-scene Domain Generalization in Multi-agent Collaborative Bird's Eye View Segmentation for Connected and Autonomous Driving [49.03947018718156]
  We propose a unified domain generalization framework to be utilized during the training and inference stages of collaborative perception. We also introduce an intra-system domain alignment mechanism to reduce or potentially eliminate the domain discrepancy among connected and autonomous vehicles.
  arXiv  Detail & Related papers  (2023-11-28T12:52:49Z)
• Foresee What You Will Learn: Data Augmentation for Domain Generalization in Non-Stationary Environments [14.344721944207599]
  Existing domain generalization aims to learn a generalizable model to perform well even on unseen domains. We propose Directional Domain Augmentation (DDA), which simulates the unseen target features by mapping source data as augmentations through a domain transformer. We evaluate the proposed method on both synthetic datasets and realworld datasets, and empirical results show that our approach can outperform other existing methods.
  arXiv  Detail & Related papers  (2023-01-19T01:51:37Z)
• Stagewise Unsupervised Domain Adaptation with Adversarial Self-Training for Road Segmentation of Remote Sensing Images [93.50240389540252]
  Road segmentation from remote sensing images is a challenging task with wide ranges of application potentials. We propose a novel stagewise domain adaptation model called RoadDA to address the domain shift (DS) issue in this field. Experiment results on two benchmarks demonstrate that RoadDA can efficiently reduce the domain gap and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-08-28T09:29:14Z)
• Multi-path Neural Networks for On-device Multi-domain Visual Classification [55.281139434736254]
  This paper proposes a novel approach to automatically learn a multi-path network for multi-domain visual classification on mobile devices. The proposed multi-path network is learned from neural architecture search by applying one reinforcement learning controller for each domain to select the best path in the super-network created from a MobileNetV3-like search space. The determined multi-path model selectively shares parameters across domains in shared nodes while keeping domain-specific parameters within non-shared nodes in individual domain paths.
  arXiv  Detail & Related papers  (2020-10-10T05:13:49Z)
• Towards Adaptive Semantic Segmentation by Progressive Feature Refinement [16.40758125170239]
  We propose an innovative progressive feature refinement framework, along with domain adversarial learning to boost the transferability of segmentation networks. As a result, the segmentation models trained with source domain images can be transferred to a target domain without significant performance degradation.
  arXiv  Detail & Related papers  (2020-09-30T04:17:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.