Envision4D: Envisioning Visual Futures via Feed-forward 4D Gaussian Splatting for Autonomous Driving

• URL: http://arxiv.org/abs/2606.10656v1
• Date: Tue, 09 Jun 2026 10:04:38 GMT
• Title: Envision4D: Envisioning Visual Futures via Feed-forward 4D Gaussian Splatting for Autonomous Driving
• Abstract summary: Envision4D is a fully self-supervised feed-forward framework for pose-free future extrapolation.<n>Future Pose Prediction module infers future camera parameters via an iterative denoising process.<n>In-layer Temporal Attention and Conditioned Motion Lifting transform the highly uncertain extrapolation process into robust mappings.
• Score: 22.824000058639125
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Forecasting the future evolution of dynamic scenes is crucial in autonomous driving. However, existing feed-forward paradigms are primarily designed for interpolation. When extended to future extrapolation, they suffer from ghosting artifacts under large displacements and are constrained by simplified motion assumptions or strict future priors. To overcome these challenges, we propose Envision4D, a fully self-supervised feed-forward framework for pose-free future extrapolation. Specifically, we introduce a Future Pose Prediction module that infers future camera parameters via an iterative denoising process. Furthermore, to capture non-linear dynamics, we propose In-layer Temporal Attention and employ Conditioned Motion Lifting, which transforms the highly uncertain extrapolation process into robust relational mappings. Finally, a Progressive Training Strategy is utilized to stabilize unsupervised motion learning against error accumulation. Extensive experiments demonstrate that Envision4D achieves state-of-the-art performance, significantly outperforming existing methods in future view synthesis.

Related papers

• Drive-HWM: Hierarchical World Models for Dynamic-Latent Guided Autonomous Driving [78.83165764484562]
  Drive-HWM is a hierarchical slow--fast world modeling framework that organizes future representation prediction and action generation at complementary temporal scales.<n>Next-frame prediction encourages the fast model to capture imminent scene evolution, while one-step action generation allows decisions to be continuously updated as new observations arrive.
  arXiv  Detail & Related papers  (2026-09-03T09:21:19Z)
• OWMDrive: Causality-Aware End-to-End Autonomous Driving via 4D Occupancy World Model [2.4196046716427393]
  OWMDrive is a generative end-to-end driving framework built upon an Occupancy World Model for multi-step 3D occupancy forecasting.<n>Conditioned on both current observations and predicted future states, OWMDrive iteratively refines trajectory candidates to generate a reinforced driving trajectory.
  arXiv  Detail & Related papers  (2026-06-29T15:05:41Z)
• IDOL: Inverse-Dynamics-Guided Future Prediction for End-to-End Autonomous Driving [6.515607168928518]
  IDOL is an inverse-dynamics-guided future prediction framework for world-model-based end-to-end planning in latent BEV space.<n>IDOL first predicts multiple future latent scene states with a BEV world model, then applies an inverse dynamics model to adjacent latent futures.<n>These inverse-dynamics-derived signals are used to optimize the planned trajectory, turning future forecasting from passive scene anticipation into actionable planning guidance.
  arXiv  Detail & Related papers  (2026-05-29T16:05:42Z)
• ELAN4D: Embodiment-Centric 4D Supervision for Vision-Language-Action Models via Plug-and-Play Adaptation [63.617951135459016]
  Vision-Language-Action (VLA) models have shown promise for robotic manipulation, yet most existing policies reactively by directly regressing actions from current observations.<n>We propose ELAN4D, an embodiment-centric, 4D-aware training framework that enhances policies with future robot key tracks as predictive-temporal supervision.
  arXiv  Detail & Related papers  (2026-05-28T19:03:30Z)
• From Imagined Futures to Executable Actions: Mixture of Latent Actions for Robot Manipulation [88.39072412680633]
  We propose MoLA, a control-oriented interface that transforms imagined future videos into executable representations.<n>We evaluate our approach on simulated benchmarks and real-world robot manipulation tasks.
  arXiv  Detail & Related papers  (2026-05-12T14:15:16Z)
• FutureVLA: Joint Visuomotor Prediction for Vision-Language-Action Model [73.03346643967309]
  We argue that effective joint visuomotor predictive modeling requires both temporal continuity and visually-conditioned supervision decoupling.<n>FutureVLA is designed to extract joint visuomotor embeddings by first decoupling visual and motor information.<n>In the post-training stage, we employ a latent embeddings alignment strategy, enabling diverse downstream VLA models to internalize these temporal priors.
  arXiv  Detail & Related papers  (2026-03-11T12:39:55Z)
• GeoMotion: Rethinking Motion Segmentation via Latent 4D Geometry [61.24189040578178]
  We propose a fully learning-based approach that directly infers moving objects from latent feature representations via attention mechanisms.<n>Our key insight is to bypass explicit correspondence estimation and instead let the model learn to implicitly disentangle object and camera motion.<n>Our approach achieves state-of-the-art motion segmentation performance with high efficiency.
  arXiv  Detail & Related papers  (2026-02-25T11:36:33Z)
• FutureX: Enhance End-to-End Autonomous Driving via Latent Chain-of-Thought World Model [103.2513470454204]
  FutureX is a pipeline that enhances end-to-end planners to perform complex motion planning via future scene latent reasoning and trajectory refinement.<n>FutureX enhances existing methods by producing more rational motion plans and fewer collisions without compromising efficiency.
  arXiv  Detail & Related papers  (2025-12-12T02:12:49Z)
• TransDiffuser: Diverse Trajectory Generation with Decorrelated Multi-modal Representation for End-to-end Autonomous Driving [20.679370777762987]
  We propose TransDiffuser, an encoder-decoder based generative trajectory planning model.<n>We exploit a simple yet effective multi-modal representation decorrelation optimization mechanism during the denoising process.<n>TransDiffuser achieves the PDMS of 94.85 on the closed-loop planning-oriented benchmark NAVSIM.
  arXiv  Detail & Related papers  (2025-05-14T12:10:41Z)
• EnerVerse: Envisioning Embodied Future Space for Robotics Manipulation [55.26713167507132]
  We introduce a generative robotics foundation model that constructs and interprets embodied spaces.<n>EnerVerse employs an autoregressive video diffusion framework to predict future embodied spaces from instructions, enhanced by a sparse context memory for long-term reasoning.<n>We present EnerVerse-D, a data engine pipeline combining the generative model with 4D Gaussian Splatting, forming a self-reinforcing data loop to reduce the sim-to-real gap.
  arXiv  Detail & Related papers  (2025-01-03T17:00:33Z)
• TransFusion: A Practical and Effective Transformer-based Diffusion Model for 3D Human Motion Prediction [1.8923948104852863]
  We propose TransFusion, an innovative and practical diffusion-based model for 3D human motion prediction. Our model leverages Transformer as the backbone with long skip connections between shallow and deep layers. In contrast to prior diffusion-based models that utilize extra modules like cross-attention and adaptive layer normalization, we treat all inputs, including conditions, as tokens to create a more lightweight model.
  arXiv  Detail & Related papers  (2023-07-30T01:52:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.