WorldSplat: Gaussian-Centric Feed-Forward 4D Scene Generation for Autonomous Driving

• URL: http://arxiv.org/abs/2509.23402v1
• Date: Sat, 27 Sep 2025 16:47:44 GMT
• Title: WorldSplat: Gaussian-Centric Feed-Forward 4D Scene Generation for Autonomous Driving
• Authors: Ziyue Zhu, Zhanqian Wu, Zhenxin Zhu, Lijun Zhou, Haiyang Sun, Bing Wan, Kun Ma, Guang Chen, Hangjun Ye, Jin Xie, jian Yang,
• Abstract summary: We propose textbfWorldSplat, a novel feed-forward framework for 4D driving-scene generation.<n>Our approach effectively generates consistent multi-track videos through two key steps.<n>Experiments conducted on benchmark datasets demonstrate that textbfWorldSplat effectively generates high-fidelity, temporally and spatially consistent novel view driving videos.
• Score: 21.778139777889397
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in driving-scene generation and reconstruction have demonstrated significant potential for enhancing autonomous driving systems by producing scalable and controllable training data. Existing generation methods primarily focus on synthesizing diverse and high-fidelity driving videos; however, due to limited 3D consistency and sparse viewpoint coverage, they struggle to support convenient and high-quality novel-view synthesis (NVS). Conversely, recent 3D/4D reconstruction approaches have significantly improved NVS for real-world driving scenes, yet inherently lack generative capabilities. To overcome this dilemma between scene generation and reconstruction, we propose \textbf{WorldSplat}, a novel feed-forward framework for 4D driving-scene generation. Our approach effectively generates consistent multi-track videos through two key steps: ((i)) We introduce a 4D-aware latent diffusion model integrating multi-modal information to produce pixel-aligned 4D Gaussians in a feed-forward manner. ((ii)) Subsequently, we refine the novel view videos rendered from these Gaussians using a enhanced video diffusion model. Extensive experiments conducted on benchmark datasets demonstrate that \textbf{WorldSplat} effectively generates high-fidelity, temporally and spatially consistent multi-track novel view driving videos.

Related papers

• Diff4Splat: Controllable 4D Scene Generation with Latent Dynamic Reconstruction Models [79.06910348413861]
  We introduce Diff4Splat, a feed-forward method that synthesizes controllable and explicit 4D scenes from a single image.<n>Given a single input image, a camera trajectory, and an optional text prompt, Diff4Splat directly predicts a deformable 3D Gaussian field that encodes appearance, geometry, and motion.
  arXiv  Detail & Related papers  (2025-11-01T11:16:25Z)
• CVD-STORM: Cross-View Video Diffusion with Spatial-Temporal Reconstruction Model for Autonomous Driving [26.379817613036597]
  CVD-STORM is a cross-view video diffusion model utilizing a spatial-temporal reconstruction Variational Autoencoder (VAE)<n>Our approach first fine-tunes the VAE with an auxiliary 4D reconstruction task, enhancing its ability to encode 3D structures and temporal dynamics.<n> Experimental results demonstrate that our model achieves substantial improvements in both FID and FVD metrics.
  arXiv  Detail & Related papers  (2025-10-09T08:41:58Z)
• 4D Driving Scene Generation With Stereo Forcing [62.47705572424127]
  Current generative models struggle to synthesize dynamic 4D driving scenes that simultaneously support temporal extrapolation and spatial novel view synthesis (NVS) without per-scene optimization.<n>We present PhiGenesis, a unified framework for 4D scene generation that extends video generation techniques with geometric and temporal consistency.
  arXiv  Detail & Related papers  (2025-09-24T15:37:17Z)
• DreamDrive: Generative 4D Scene Modeling from Street View Images [55.45852373799639]
  We present DreamDrive, a 4D spatial-temporal scene generation approach that combines the merits of generation and reconstruction.<n>Specifically, we leverage the generative power of video diffusion models to synthesize a sequence of visual references.<n>We then render 3D-consistent driving videos via Gaussian splatting.
  arXiv  Detail & Related papers  (2024-12-31T18:59:57Z)
• DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representation [32.19534057884047]
  We introduce DriveDreamer4D, which enhances 4D driving scene representation leveraging world model priors. To our knowledge, DriveDreamer4D is the first to utilize video generation models for improving 4D reconstruction in driving scenarios.
  arXiv  Detail & Related papers  (2024-10-17T14:07:46Z)
• ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis [63.169364481672915]
  We propose textbfViewCrafter, a novel method for synthesizing high-fidelity novel views of generic scenes from single or sparse images. Our method takes advantage of the powerful generation capabilities of video diffusion model and the coarse 3D clues offered by point-based representation to generate high-quality video frames.
  arXiv  Detail & Related papers  (2024-09-03T16:53:19Z)
• EG4D: Explicit Generation of 4D Object without Score Distillation [105.63506584772331]
  DG4D is a novel framework that generates high-quality and consistent 4D assets without score distillation. Our framework outperforms the baselines in generation quality by a considerable margin.
  arXiv  Detail & Related papers  (2024-05-28T12:47:22Z)
• Diffusion4D: Fast Spatial-temporal Consistent 4D Generation via Video Diffusion Models [116.31344506738816]
  We present a novel framework, textbfDiffusion4D, for efficient and scalable 4D content generation. We develop a 4D-aware video diffusion model capable of synthesizing orbital views of dynamic 3D assets. Our method surpasses prior state-of-the-art techniques in terms of generation efficiency and 4D geometry consistency.
  arXiv  Detail & Related papers  (2024-05-26T17:47:34Z)
• SC4D: Sparse-Controlled Video-to-4D Generation and Motion Transfer [57.506654943449796]
  We propose an efficient, sparse-controlled video-to-4D framework named SC4D that decouples motion and appearance. Our method surpasses existing methods in both quality and efficiency. We devise a novel application that seamlessly transfers motion onto a diverse array of 4D entities.
  arXiv  Detail & Related papers  (2024-04-04T18:05:18Z)
• STAG4D: Spatial-Temporal Anchored Generative 4D Gaussians [36.83603109001298]
  STAG4D is a novel framework that combines pre-trained diffusion models with dynamic 3D Gaussian splatting for high-fidelity 4D generation. We show that our method outperforms prior 4D generation works in rendering quality, spatial-temporal consistency, and generation robustness.
  arXiv  Detail & Related papers  (2024-03-22T04:16:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.