GEM: Gaussian Evolution Model for Occupancy Forecasting and Motion Planning

• URL: http://arxiv.org/abs/2605.17682v1
• Date: Sun, 17 May 2026 22:52:53 GMT
• Title: GEM: Gaussian Evolution Model for Occupancy Forecasting and Motion Planning
• Abstract summary: We propose a non-autoregressive occupancy world model for future semantic occupancy forecasting and motion planning.<n>We show that GEM achieves state-of-the-art future semantic occupancy forecasting and strong motion planning performance.
• Score: 13.877624821024535
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Future 3D semantic occupancy forecasting and motion planning are central to autonomous driving, as they require models to reason about how surrounding scenes evolve and how the ego vehicle should act. Existing occupancy world models commonly discretize scenes into latent embeddings, volumetric features, or quantized tokens, and forecast future states through fixed-step autoregressive generation. This limits temporal flexibility, obscures scene evolution, accumulates errors over long horizons, and poorly matches the continuous-time dynamics of real driving scenes. We propose GEM, a Gaussian Evolution Model for non-autoregressive occupancy world modeling, where driving scenes are represented as explicit continuous 4D Gaussian primitives with learned dynamics. Instead of rolling out future occupancy states step by step, GEM directly queries the Gaussian world representation at arbitrary timestamps and splats the corresponding conditional 3D Gaussians into semantic occupancy volumes. This enables efficient forecasting over the full horizon while retaining a compact and interpretable scene representation. By decoupling spatial geometry, temporal support, and primitive motion, GEM makes the predicted world easier to inspect, as each primitive's evolution can be followed continuously over time. The same representation also supports motion planning by predicting future ego trajectories from the learned Gaussian world. Extensive experiments show that GEM achieves state-of-the-art future semantic occupancy forecasting and strong motion planning performance, while providing flexible temporal querying.

Related papers

• Drive-HWM: Hierarchical World Models for Dynamic-Latent Guided Autonomous Driving [78.83165764484562]
  Drive-HWM is a hierarchical slow--fast world modeling framework that organizes future representation prediction and action generation at complementary temporal scales.<n>Next-frame prediction encourages the fast model to capture imminent scene evolution, while one-step action generation allows decisions to be continuously updated as new observations arrive.
  arXiv  Detail & Related papers  (2026-09-03T09:21:19Z)
• GaussianDream++: Efficient 3D Gaussian World Modeling for Robotic Manipulation [55.47130289185285]
  Vision-Language-Action (VLA) policies have advanced language-conditioned robotic manipulation.<n>We present textbfMethodname, a compact, policy-native extension that inserts textbfWorld State Tokens and textbfWorld Prediction Tokens directly into the VLA backbone.
  arXiv  Detail & Related papers  (2026-08-26T11:41:55Z)
• Sensorimotor World Models: Perception for Action via Inverse Dynamics [59.121736411156384]
  We introduce a sensorimotor world model trained end-to-end with inverse dynamics regularization.<n>It prevents representation collapse and induces action-aligned representations.<n>It learns compact, interpretable latent spaces and enables competitive planning performance across simple 2D and 3D control tasks.
  arXiv  Detail & Related papers  (2026-06-18T11:25:16Z)
• Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation [66.7879424097418]
  We present WorldDrive, a holistic framework that couples scene generation and real-time planning via unifying vision and motion representation.<n>A simple interaction between motion representation, visual representation, and ego status can generate high-quality, multi-modal trajectories.<n>Experiments on the NAVSIM, NAVSIM-v2, and nuScenes benchmarks demonstrate that WorldDrive achieves leading planning performance among vision-only methods.
  arXiv  Detail & Related papers  (2026-03-16T07:59:39Z)
• VGGT-World: Transforming VGGT into an Autoregressive Geometry World Model [59.789011777899965]
  VGGT-World is a geometry world model that side-steps video generation entirely and instead forecasts the temporal evolution of frozen geometry-foundation-model features.<n>We show that VGGT-World significantly outperforms the strongest baselines in depth forecasting while running 3.6-5 times faster with only 0.43B trainable parameters.
  arXiv  Detail & Related papers  (2026-03-13T04:56:43Z)
• EgoMoD: Predicting Global Maps of Dynamics from Local Egocentric Observations [3.5831173106587393]
  Maps of Dynamics (MoDs) offer a structured representation of motion tendencies in space useful for long-term global planning.<n>We introduce EgoMoD, the first approach that learns to predict future MoDs directly from short egocentric video clips collected during robot operation.<n>Our method learns to infer environment-wide motion tendencies from local dynamic cues using a video- and pose-conditioned architecture trained with MoDs computed from external observations as privileged supervision.
  arXiv  Detail & Related papers  (2026-02-26T09:56:21Z)
• Space-Time Forecasting of Dynamic Scenes with Motion-aware Gaussian Grouping [47.282424020983996]
  MoGaF is a framework for long-term scene extrapolation built upon the 4D Gaussian Splatting representation.<n>A lightweight forecasting module predicts future motion, enabling realistic and temporally stable scene evolution.<n>Experiments on synthetic and real-world datasets demonstrate that MoGaF consistently outperforms existing baselines in rendering quality, motion plausibility, and long-term forecasting stability.
  arXiv  Detail & Related papers  (2026-02-25T08:04:07Z)
• OccTENS: 3D Occupancy World Model via Temporal Next-Scale Prediction [25.630452373274636]
  OccTENS is a generative occupancy world model that enables controllable, high-fidelity long-term occupancy generation.<n>We reformulate the occupancy world model as a temporal next-scale prediction (TENS) task.<n>OccTENS outperforms the state-of-the-art method with both higher occupancy quality and faster inference time.
  arXiv  Detail & Related papers  (2025-09-04T05:06:47Z)
• GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Prediction [67.81475355852997]
  3D occupancy prediction is important for autonomous driving due to its comprehensive perception of the surroundings.<n>We propose a world-model-based framework to exploit the scene evolution for perception.<n>Our framework improves the performance of the single-frame counterpart by over 2% in mIoU without introducing additional computations.
  arXiv  Detail & Related papers  (2024-12-13T18:59:54Z)
• GaussianPrediction: Dynamic 3D Gaussian Prediction for Motion Extrapolation and Free View Synthesis [71.24791230358065]
  We introduce a novel framework that empowers 3D Gaussian representations with dynamic scene modeling and future scenario synthesis. GaussianPrediction can forecast future states from any viewpoint, using video observations of dynamic scenes. Our framework shows outstanding performance on both synthetic and real-world datasets, demonstrating its efficacy in predicting and rendering future environments.
  arXiv  Detail & Related papers  (2024-05-30T06:47:55Z)
• AMP: Autoregressive Motion Prediction Revisited with Next Token Prediction for Autonomous Driving [59.94343412438211]
  We introduce the GPT style next token motion prediction into motion prediction. Different from language data which is composed of homogeneous units -words, the elements in the driving scene could have complex spatial-temporal and semantic relations. We propose to adopt three factorized attention modules with different neighbors for information aggregation and different position encoding styles to capture their relations.
  arXiv  Detail & Related papers  (2024-03-20T06:22:37Z)
• GenAD: Generative End-to-End Autonomous Driving [13.332272121018285]
  GenAD is a generative framework that casts autonomous driving into a generative modeling problem. We propose an instance-centric scene tokenizer that first transforms the surrounding scenes into map-aware instance tokens. We then employ a variational autoencoder to learn the future trajectory distribution in a structural latent space for trajectory prior modeling.
  arXiv  Detail & Related papers  (2024-02-18T08:21:05Z)
• A Spatio-temporal Transformer for 3D Human Motion Prediction [39.31212055504893]
  We propose a Transformer-based architecture for the task of generative modelling of 3D human motion. We empirically show that this effectively learns the underlying motion dynamics and reduces error accumulation over time observed in auto-gressive models.
  arXiv  Detail & Related papers  (2020-04-18T19:49:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.