4DGS-WAM: Bridging Past and Future with an Object-Centric World Action Model based on 4D Gaussian Splatting

• URL: http://arxiv.org/abs/2608.25956v1
• Date: Wed, 26 Aug 2026 16:16:57 GMT
• Title: 4DGS-WAM: Bridging Past and Future with an Object-Centric World Action Model based on 4D Gaussian Splatting
• Abstract summary: Current world action models (WAMs) typically operate on 2D visual data.<n>We leverage an explicit 4D Gaussian Splatting (4DGS) representation that separately models dynamic objects and the static background of a scene.<n>4DGS-WAM lifts 2D observations into a persistent 4D representation so that previously observed static content can be reused during future prediction.
• Score: 72.02343855552051
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Current world action models (WAMs) typically operate on 2D visual data. These models can achieve exceptional visual quality, but they lack explicit spatial structure for individual objects and repeatedly process redundant background content. Although point clouds can represent the world in 3D space, they can be difficult to align and accumulate across viewpoints. In this paper, we leverage an explicit 4D Gaussian Splatting (4DGS) representation that separately models dynamic objects and the static background of a scene. For dynamic objects, we use a policy model to predict future actor actions and a world model to predict transformations of their observed Gaussian splats. The static background need not be regenerated for future states, as much of it has already been observed in past frames. This forms an object-centric world action model, which we name 4DGS-WAM. It lifts 2D observations into a persistent 4D representation so that previously observed static content can be reused during future prediction. Future-state extrapolation can then focus on modeling the evolution of dynamic objects. Experiments on KITTI-MOT evaluate short-horizon prediction and past reconstruction.

Related papers

• Future Dynamic 3D Reconstruction: A 3D World Model with Disentangled Ego-Motion [69.61280478695205]
  We propose FR3D, a world model that predicts a persistent 3D latent representation for future dynamic 3D reconstruction.<n>Experiments demonstrate FR3D's strong performance for future dynamic 3D reconstruction from monocular observations across multiple datasets, even 2 seconds into the future.
  arXiv  Detail & Related papers  (2026-06-16T17:59:46Z)
• StemVLA:An Open-Source Vision-Language-Action Model with Future 3D Spatial Geometry Knowledge and 4D Historical Representation [6.0744834626758495]
  StemVLA is a novel framework that explicitly incorporates both future-oriented 3D spatial knowledge and historical 4D representations into action prediction.<n>We show that StemVLA significantly improves long-horizon task success and state-of-the-art performance on the CALVIN ABC-D benchmark [46], achieving an average sequence length of XXX.
  arXiv  Detail & Related papers  (2026-02-27T06:43:37Z)
• MVISTA-4D: View-Consistent 4D World Model with Test-Time Action Inference for Robotic Manipulation [27.70398018267795]
  This work proposes a novel embodied 4D world model that enables geometrically consistent, arbitrary-view RGBD generation.<n>Experiments on three datasets demonstrate strong performance on both 4D scene generation and downstream manipulation.
  arXiv  Detail & Related papers  (2026-02-10T15:19:17Z)
• ObjectForesight: Predicting Future 3D Object Trajectories from Human Videos [48.24897274501108]
  We introduce a 3D object-centric dynamics model that predicts future 6-DoF poses and trajectories of rigid objects from short egocentric video sequences.<n>Unlike conventional world or dynamics models that operate in pixel or latent space, ObjectForesight represents the world explicitly in 3D at the object level.<n>We show that ObjectForesight achieves significant gains in accuracy, geometric consistency, and generalization to unseen objects and scenes.
  arXiv  Detail & Related papers  (2026-01-08T18:58:08Z)
• TesserAct: Learning 4D Embodied World Models [66.8519958275311]
  We learn a 4D world model by training on RGB-DN (RGB, Depth, and Normal) videos.<n>This not only surpasses traditional 2D models by incorporating detailed shape, configuration, and temporal changes into their predictions, but also allows us to effectively learn accurate inverse dynamic models for an embodied agent.
  arXiv  Detail & Related papers  (2025-04-29T17:59:30Z)
• GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Prediction [67.81475355852997]
  3D occupancy prediction is important for autonomous driving due to its comprehensive perception of the surroundings.<n>We propose a world-model-based framework to exploit the scene evolution for perception.<n>Our framework improves the performance of the single-frame counterpart by over 2% in mIoU without introducing additional computations.
  arXiv  Detail & Related papers  (2024-12-13T18:59:54Z)
• UrbanGS: Semantic-Guided Gaussian Splatting for Urban Scene Reconstruction [86.4386398262018]
  UrbanGS uses 2D semantic maps and an existing dynamic Gaussian approach to distinguish static objects from the scene.<n>For potentially dynamic objects, we aggregate temporal information using learnable time embeddings.<n>Our approach outperforms state-of-the-art methods in reconstruction quality and efficiency.
  arXiv  Detail & Related papers  (2024-12-04T16:59:49Z)
• 3D-IntPhys: Towards More Generalized 3D-grounded Visual Intuitive Physics under Challenging Scenes [68.66237114509264]
  We present a framework capable of learning 3D-grounded visual intuitive physics models from videos of complex scenes with fluids. We show our model can make long-horizon future predictions by learning from raw images and significantly outperforms models that do not employ an explicit 3D representation space.
  arXiv  Detail & Related papers  (2023-04-22T19:28:49Z)
• 3D-OES: Viewpoint-Invariant Object-Factorized Environment Simulators [24.181604511269096]
  We propose an action-conditioned dynamics model that predicts scene changes caused by object and agent interactions in a viewpoint-in 3D neural scene representation space. In this space, objects do not interfere with one another and their appearance persists over time and across viewpoints. We show our model generalizes well its predictions across varying number and appearances of interacting objects as well as across camera viewpoints.
  arXiv  Detail & Related papers  (2020-11-12T16:15:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.