DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving

• URL: http://arxiv.org/abs/2411.15139v1
• Date: Fri, 22 Nov 2024 18:59:47 GMT
• Title: DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving
• Authors: Bencheng Liao, Shaoyu Chen, Haoran Yin, Bo Jiang, Cheng Wang, Sixu Yan, Xinbang Zhang, Xiangyu Li, Ying Zhang, Qian Zhang, Xinggang Wang,
• Abstract summary: Diffusion model has emerged as a powerful generative technique for robotic policy learning. We propose a novel truncated diffusion policy that incorporates prior multi-mode anchors and truncates the diffusion schedule. The proposed model, DiffusionDrive, demonstrates 10$times$ reduction in denoising steps compared to vanilla diffusion policy.
• Score: 38.867860153968394
• License:
• Abstract: Recently, the diffusion model has emerged as a powerful generative technique for robotic policy learning, capable of modeling multi-mode action distributions. Leveraging its capability for end-to-end autonomous driving is a promising direction. However, the numerous denoising steps in the robotic diffusion policy and the more dynamic, open-world nature of traffic scenes pose substantial challenges for generating diverse driving actions at a real-time speed. To address these challenges, we propose a novel truncated diffusion policy that incorporates prior multi-mode anchors and truncates the diffusion schedule, enabling the model to learn denoising from anchored Gaussian distribution to the multi-mode driving action distribution. Additionally, we design an efficient cascade diffusion decoder for enhanced interaction with conditional scene context. The proposed model, DiffusionDrive, demonstrates 10$\times$ reduction in denoising steps compared to vanilla diffusion policy, delivering superior diversity and quality in just 2 steps. On the planning-oriented NAVSIM dataset, with the aligned ResNet-34 backbone, DiffusionDrive achieves 88.1 PDMS without bells and whistles, setting a new record, while running at a real-time speed of 45 FPS on an NVIDIA 4090. Qualitative results on challenging scenarios further confirm that DiffusionDrive can robustly generate diverse plausible driving actions. Code and model will be available at https://github.com/hustvl/DiffusionDrive.

Related papers

• Latent Weight Diffusion: Generating Policies from Trajectories [13.404962654299041]
  Latent Weight Diffusion (LWD) is a method that uses diffusion to learn a distribution over policies for robotic tasks, rather than over trajectories. LWD achieves a higher success rate compared to a vanilla multi-task policy, while using models up to 18x smaller during inference.
  arXiv  Detail & Related papers  (2024-10-17T21:30:29Z)
• Intention-aware Denoising Diffusion Model for Trajectory Prediction [14.524496560759555]
  Trajectory prediction is an essential component in autonomous driving, particularly for collision avoidance systems. We propose utilizing the diffusion model to generate the distribution of future trajectories. We propose an Intention-aware denoising Diffusion Model (IDM) Our methods achieve state-of-the-art results, with an FDE of 13.83 pixels on the SDD dataset and 0.36 meters on the ETH/UCY dataset.
  arXiv  Detail & Related papers  (2024-03-14T09:05:25Z)
• Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous Driving and Zero-Shot Instruction Following [21.81411085058986]
  Reward-gradient guided denoising generates trajectories that maximize both a differentiable reward function and the likelihood under the data distribution captured by a diffusion model. We propose DiffusionES, a method that combines gradient-free optimization with trajectory denoising. We show that DiffusionES achieves state-of-the-art performance on nuPlan, an established closed-loop planning benchmark for autonomous driving.
  arXiv  Detail & Related papers  (2024-02-09T17:18:33Z)
• Guided Diffusion from Self-Supervised Diffusion Features [49.78673164423208]
  Guidance serves as a key concept in diffusion models, yet its effectiveness is often limited by the need for extra data annotation or pretraining. We propose a framework to extract guidance from, and specifically for, diffusion models.
  arXiv  Detail & Related papers  (2023-12-14T11:19:11Z)
• EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generation [57.539634387672656]
  Current state-of-the-art generative diffusion models have produced impressive results but struggle to achieve fast generation without sacrificing quality. We introduce Efficient Motion Diffusion Model (EMDM) for fast and high-quality human motion generation.
  arXiv  Detail & Related papers  (2023-12-04T18:58:38Z)
• GDTS: Goal-Guided Diffusion Model with Tree Sampling for Multi-Modal Pedestrian Trajectory Prediction [15.731398013255179]
  We propose a novel Goal-Guided Diffusion Model with Tree Sampling for multi-modal trajectory prediction. A two-stage tree sampling algorithm is presented, which leverages common features to reduce the inference time and improve accuracy for multi-modal prediction. Experimental results demonstrate that our proposed framework achieves comparable state-of-the-art performance with real-time inference speed in public datasets.
  arXiv  Detail & Related papers  (2023-11-25T03:55:06Z)
• Semi-Implicit Denoising Diffusion Models (SIDDMs) [50.30163684539586]
  Existing models such as Denoising Diffusion Probabilistic Models (DDPM) deliver high-quality, diverse samples but are slowed by an inherently high number of iterative steps. We introduce a novel approach that tackles the problem by matching implicit and explicit factors. We demonstrate that our proposed method obtains comparable generative performance to diffusion-based models and vastly superior results to models with a small number of sampling steps.
  arXiv  Detail & Related papers  (2023-06-21T18:49:22Z)
• Modiff: Action-Conditioned 3D Motion Generation with Denoising Diffusion Probabilistic Models [58.357180353368896]
  We propose a conditional paradigm that benefits from the denoising diffusion probabilistic model (DDPM) to tackle the problem of realistic and diverse action-conditioned 3D skeleton-based motion generation. We are a pioneering attempt that uses DDPM to synthesize a variable number of motion sequences conditioned on a categorical action.
  arXiv  Detail & Related papers  (2023-01-10T13:15:42Z)
• Truncated Diffusion Probabilistic Models and Diffusion-based Adversarial Auto-Encoders [137.1060633388405]
  Diffusion-based generative models learn how to generate the data by inferring a reverse diffusion chain. We propose a faster and cheaper approach that adds noise not until the data become pure random noise. We show that the proposed model can be cast as an adversarial auto-encoder empowered by both the diffusion process and a learnable implicit prior.
  arXiv  Detail & Related papers  (2022-02-19T20:18:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.