HMPDM: A Diffusion Model for Driving Video Prediction with Historical Motion Priors

• URL: http://arxiv.org/abs/2603.27371v1
• Date: Sat, 28 Mar 2026 18:37:08 GMT
• Title: HMPDM: A Diffusion Model for Driving Video Prediction with Historical Motion Priors
• Abstract summary: This paper introduces the historical motion priors-informed diffusion model (HMPDM), a video prediction model that leverages historical motion priors to enhance motion understanding and temporal coherence.<n>Extensive experiments on the Cityscapes and KITTI benchmarks demonstrate that HMPDM outperforms state-of-the-art video prediction methods with efficiency.
• Score: 8.987844576502054
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video prediction is a useful function for autonomous driving, enabling intelligent vehicles to reliably anticipate how driving scenes will evolve and thereby supporting reasoning and safer planning. However, existing models are constrained by multi-stage training pipelines and remain insufficient in modeling the diverse motion patterns in real driving scenes, leading to degraded temporal consistency and visual quality. To address these challenges, this paper introduces the historical motion priors-informed diffusion model (HMPDM), a video prediction model that leverages historical motion priors to enhance motion understanding and temporal coherence. The proposed deep learning system introduces three key designs: (i) a Temporal-aware Latent Conditioning (TaLC) module for implicit historical motion injection; (ii) a Motion-aware Pyramid Encoder (MaPE) for multi-scale motion representation; (iii) a Self-Conditioning (SC) strategy for stable iterative denoising. Extensive experiments on the Cityscapes and KITTI benchmarks demonstrate that HMPDM outperforms state-of-the-art video prediction methods with efficiency, achieving a 28.2% improvement in FVD on Cityscapes under the same monocular RGB input configuration setting. The implementation codes are publicly available at https://github.com/KELISBU/HMPDM.

Related papers

• Drive-HWM: Hierarchical World Models for Dynamic-Latent Guided Autonomous Driving [78.83165764484562]
  Drive-HWM is a hierarchical slow--fast world modeling framework that organizes future representation prediction and action generation at complementary temporal scales.<n>Next-frame prediction encourages the fast model to capture imminent scene evolution, while one-step action generation allows decisions to be continuously updated as new observations arrive.
  arXiv  Detail & Related papers  (2026-09-03T09:21:19Z)
• UNIVERSE: Unified Video Action Models for Autonomous Driving with Flexible Mask-Modulated Modality Generation [23.675212404889805]
  World Action Models (WAMs) have shown strong potential for improving action generalization in autonomous driving.<n>Existing cascaded or dual-DiT designs separate video imagination from action prediction.<n>We propose UNIVERSE, a unified video-action model built upon a single mask-modulated Transformer.
  arXiv  Detail & Related papers  (2026-07-06T14:18:18Z)
• DriveWAM: Video Generative Priors Enable Scalable World-Action Modeling for Autonomous Driving [19.231942908215174]
  We present DriveWAM, a driving world-action model that adapts a pretrained video diffusion transformer into an autoregressive video-action policy.<n>DriveWAM organizes video and action streams into a unified temporal token sequence and trains them under a joint flow-matching objective.<n>Experiments on NAVSIM and the PhysicalAI-Autonomous-Vehicles benchmark show that DriveWAM achieves strong planning performance.
  arXiv  Detail & Related papers  (2026-05-27T14:36:26Z)
• DriveVA: Video Action Models are Zero-Shot Drivers [23.675212404889805]
  Generalization is a central challenge in autonomous driving, as real-world scenarios require robust performance under unseen conditions.<n>Recent world-model-based planning methods have shown strong capabilities in scene understanding and multi-modal future prediction.<n>We propose DriveVA, a novel autonomous driving world model that jointly decodes future visual forecasts and action sequences in a shared latent generative process.
  arXiv  Detail & Related papers  (2026-04-05T17:43:16Z)
• Real-Time Motion-Controllable Autoregressive Video Diffusion [79.32730467857535]
  We propose AR-Drag, the first RL-enhanced few-step AR video diffusion model for real-time image-to-video generation with diverse motion control.<n>We first fine-tune a base I2V model to support basic motion control, then further improve it via reinforcement with a trajectory-based reward model.<n>Our design preserves the Markov property through a Self-Rollout learning mechanism and accelerates training by selectively denoising steps.
  arXiv  Detail & Related papers  (2025-10-09T12:17:11Z)
• FlowMo: Variance-Based Flow Guidance for Coherent Motion in Video Generation [51.110607281391154]
  FlowMo is a training-free guidance method for enhancing motion coherence in text-to-video models.<n>It estimates motion coherence by measuring the patch-wise variance across the temporal dimension and guides the model to reduce this variance dynamically during sampling.
  arXiv  Detail & Related papers  (2025-06-01T19:55:33Z)
• Generative Pre-trained Autoregressive Diffusion Transformer [74.25668109048418]
  GPDiT is a Generative Pre-trained Autoregressive Diffusion Transformer.<n>It unifies the strengths of diffusion and autoregressive modeling for long-range video synthesis.<n>It autoregressively predicts future latent frames using a diffusion loss, enabling natural modeling of motion dynamics.
  arXiv  Detail & Related papers  (2025-05-12T08:32:39Z)
• DrivingWorld: Constructing World Model for Autonomous Driving via Video GPT [33.943125216555316]
  We present DrivingWorld, a GPT-style world model for autonomous driving.<n>We propose a next-state prediction strategy to model temporal coherence between consecutive frames.<n>We also propose a novel masking strategy and reweighting strategy for token prediction to mitigate long-term drifting issues.
  arXiv  Detail & Related papers  (2024-12-27T07:44:07Z)
• DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformers [61.92571851411509]
  We introduce a multimodal driving language based on interleaved image and action tokens, and develop DrivingGPT to learn joint world modeling and planning.<n>Our DrivingGPT demonstrates strong performance in both action-conditioned video generation and end-to-end planning, outperforming strong baselines on large-scale nuPlan and NAVSIM benchmarks.
  arXiv  Detail & Related papers  (2024-12-24T18:59:37Z)
• Interactive Character Control with Auto-Regressive Motion Diffusion Models [18.727066177880708]
  We propose A-MDM (Auto-regressive Motion Diffusion Model) for real-time motion synthesis. Our conditional diffusion model takes an initial pose as input, and auto-regressively generates successive motion frames conditioned on previous frame. We introduce a suite of techniques for incorporating interactive controls into A-MDM, such as task-oriented sampling, in-painting, and hierarchical reinforcement learning.
  arXiv  Detail & Related papers  (2023-06-01T07:48:34Z)
• Conditional Temporal Variational AutoEncoder for Action Video Prediction [66.63038712306606]
  ACT-VAE predicts pose sequences for an action clips from a single input image. When connected with a plug-and-play Pose-to-Image (P2I) network, ACT-VAE can synthesize image sequences.
  arXiv  Detail & Related papers  (2021-08-12T10:59:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.