Breaking Imitation Bottlenecks: Reinforced Diffusion Powers Diverse Trajectory Generation

• URL: http://arxiv.org/abs/2507.04049v2
• Date: Tue, 05 Aug 2025 04:40:32 GMT
• Title: Breaking Imitation Bottlenecks: Reinforced Diffusion Powers Diverse Trajectory Generation
• Authors: Ziying Song, Lin Liu, Hongyu Pan, Bencheng Liao, Mingzhe Guo, Lei Yang, Yongchang Zhang, Shaoqing Xu, Caiyan Jia, Yadan Luo,
• Abstract summary: DIVER is an end-to-end autonomous driving framework that integrates reinforcement learning and diffusion-based generation.<n>We show that DIVER significantly improves trajectory diversity, effectively addressing the mode collapse problem inherent in imitation learning.
• Score: 20.106116218594266
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Most end-to-end autonomous driving methods rely on imitation learning from single expert demonstrations, often leading to conservative and homogeneous behaviors that limit generalization in complex real-world scenarios. In this work, we propose DIVER, an end-to-end driving framework that integrates reinforcement learning with diffusion-based generation to produce diverse and feasible trajectories. At the core of DIVER lies a reinforced diffusion-based generation mechanism. First, the model conditions on map elements and surrounding agents to generate multiple reference trajectories from a single ground-truth trajectory, alleviating the limitations of imitation learning that arise from relying solely on single expert demonstrations. Second, reinforcement learning is employed to guide the diffusion process, where reward-based supervision enforces safety and diversity constraints on the generated trajectories, thereby enhancing their practicality and generalization capability. Furthermore, to address the limitations of L2-based open-loop metrics in capturing trajectory diversity, we propose a novel Diversity metric to evaluate the diversity of multi-mode predictions.Extensive experiments on the closed-loop NAVSIM and Bench2Drive benchmarks, as well as the open-loop nuScenes dataset, demonstrate that DIVER significantly improves trajectory diversity, effectively addressing the mode collapse problem inherent in imitation learning.

Related papers

• MoCa: Modality-aware Continual Pre-training Makes Better Bidirectional Multimodal Embeddings [75.0617088717528]
  MoCa is a framework for transforming pre-trained VLM backbones into effective bidirectional embedding models.<n>MoCa consistently improves performance across MMEB and ViDoRe-v2 benchmarks, achieving new state-of-the-art results.
  arXiv  Detail & Related papers  (2025-06-29T06:41:00Z)
• BiTrajDiff: Bidirectional Trajectory Generation with Diffusion Models for Offline Reinforcement Learning [11.850736201059217]
  BiTrajDiff is a novel DA framework for offline Reinforcement Learning (RL)<n>We decompose the trajectory generation task into two independent yet complementary diffusion processes.<n>BiTrajDiff can efficiently leverage critical states as anchors to expand into potentially valuable yet underexplored regions.
  arXiv  Detail & Related papers  (2025-06-06T05:41:33Z)
• Continual Multimodal Contrastive Learning [70.60542106731813]
  Multimodal contrastive learning (MCL) advances in aligning different modalities and generating multimodal representations in a joint space.<n>However, a critical yet often overlooked challenge remains: multimodal data is rarely collected in a single process, and training from scratch is computationally expensive.<n>In this paper, we formulate CMCL through two specialized principles of stability and plasticity.<n>We theoretically derive a novel optimization-based method, which projects updated gradients from dual sides onto subspaces where any gradient is prevented from interfering with the previously learned knowledge.
  arXiv  Detail & Related papers  (2025-03-19T07:57:08Z)
• Disentangled World Models: Learning to Transfer Semantic Knowledge from Distracting Videos for Reinforcement Learning [93.58897637077001]
  This paper tries to learn and understand underlying semantic variations from distracting videos via offline-to-online latent distillation and flexible disentanglement constraints.<n>We pretrain the action-free video prediction model offline with disentanglement regularization to extract semantic knowledge from distracting videos.<n>For finetuning in the online environment, we exploit the knowledge from the pretrained model and introduce a disentanglement constraint to the world model.
  arXiv  Detail & Related papers  (2025-03-11T13:50:22Z)
• Learning Soft Driving Constraints from Vectorized Scene Embeddings while Imitating Expert Trajectories [16.666811573117613]
  The primary goal of motion planning is to generate safe and efficient trajectories for vehicles.<n>Traditionally, motion planning models are trained using imitation learning to mimic the behavior of human experts.<n>We propose a method that integrates constraint learning into imitation learning by extracting driving constraints from expert trajectories.
  arXiv  Detail & Related papers  (2024-12-07T18:29:28Z)
• FlickerFusion: Intra-trajectory Domain Generalizing Multi-Agent RL [19.236153474365747]
  Existing MARL approaches often rely on the restrictive assumption that the number of entities remains constant between training and inference.<n>In this paper, we tackle the challenge of intra-trajectory dynamic entity composition under zero-shot out-of-domain (OOD) generalization.<n>We propose FlickerFusion, a novel OOD generalization method that acts as a universally applicable augmentation technique for MARL backbone methods.
  arXiv  Detail & Related papers  (2024-10-21T10:57:45Z)
• Diffusing States and Matching Scores: A New Framework for Imitation Learning [16.941612670582522]
  Adversarial Imitation Learning is traditionally framed as a two-player zero-sum game between a learner and an adversarially chosen cost function.<n> diffusion models have emerged as a non-adversarial alternative to GANs that merely require training a score function via regression.<n>We show our approach outperforms both GAN-style imitation learning baselines and discriminator-free imitation learning baselines across various continuous control problems.
  arXiv  Detail & Related papers  (2024-10-17T17:59:25Z)
• Conditional Neural Expert Processes for Learning Movement Primitives from Demonstration [1.9336815376402723]
  Conditional Neural Expert Processes (CNEP) learns to assign demonstrations from different modes to distinct expert networks. CNEP does not require supervision on which mode the trajectories belong to. Our system is capable of on-the-fly adaptation to environmental changes via an online conditioning mechanism.
  arXiv  Detail & Related papers  (2024-02-13T12:52:02Z)
• Beyond DAGs: A Latent Partial Causal Model for Multimodal Learning [80.44084021062105]
  We propose a novel latent partial causal model for multimodal data, featuring two latent coupled variables, connected by an undirected edge, to represent the transfer of knowledge across modalities.<n>Under specific statistical assumptions, we establish an identifiability result, demonstrating that representations learned by multimodal contrastive learning correspond to the latent coupled variables up to a trivial transformation.<n>Experiments on a pre-trained CLIP model embodies disentangled representations, enabling few-shot learning and improving domain generalization across diverse real-world datasets.
  arXiv  Detail & Related papers  (2024-02-09T07:18:06Z)
• Divide-and-Conquer for Lane-Aware Diverse Trajectory Prediction [71.97877759413272]
  Trajectory prediction is a safety-critical tool for autonomous vehicles to plan and execute actions. Recent methods have achieved strong performances using Multi-Choice Learning objectives like winner-takes-all (WTA) or best-of-many. Our work addresses two key challenges in trajectory prediction, learning outputs, and better predictions by imposing constraints using driving knowledge.
  arXiv  Detail & Related papers  (2021-04-16T17:58:56Z)
• SMART: Simultaneous Multi-Agent Recurrent Trajectory Prediction [72.37440317774556]
  We propose advances that address two key challenges in future trajectory prediction. multimodality in both training data and predictions and constant time inference regardless of number of agents.
  arXiv  Detail & Related papers  (2020-07-26T08:17:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.