GaussianDream++: Efficient 3D Gaussian World Modeling for Robotic Manipulation
- URL: http://arxiv.org/abs/2608.25659v1
- Date: Wed, 26 Aug 2026 11:41:55 GMT
- Title: GaussianDream++: Efficient 3D Gaussian World Modeling for Robotic Manipulation
- Abstract summary: Vision-Language-Action (VLA) policies have advanced language-conditioned robotic manipulation.<n>We present textbfMethodname, a compact, policy-native extension that inserts textbfWorld State Tokens and textbfWorld Prediction Tokens directly into the VLA backbone.
- Score: 55.47130289185285
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Vision-Language-Action (VLA) policies have advanced language-conditioned robotic manipulation, yet action-imitation objectives provide only weak supervision for metric 3D structure and short-horizon physical evolution. Geometry-enhanced policies mainly improve current-scene grounding, whereas predictive policies often model future dynamics in RGB or latent spaces and may incur substantial deployment cost. GaussianDream demonstrates that training-time current Gaussian reconstruction and future Gaussian prediction provide effective 3D supervision, but its dense VGGT/TGE-based prefix jointly carries state, dynamics, and action-conditioning information. We present \textbf{\methodname}, a compact, policy-native extension that inserts \textbf{World State Tokens} and \textbf{World Prediction Tokens} directly into the VLA backbone. A training-only \textbf{World Representation Head} decodes these tokens into a Current World and coupled Future Prediction over shared Gaussian primitives, while static--dynamic factorization preserves persistent structure and focuses residual motion on interaction-relevant regions. At inference, the head, renderer, auxiliary objectives, and VGGT/TGE pathway are removed, leaving only 20 world tokens without online Gaussian decoding or rollout. \method achieves \textbf{98.6\%} on LIBERO and \textbf{87.8\%} on LIBERO-Plus, with clear gains under Camera and Layout shifts. Real-robot experiments further improve average success from 29.2\% to 52.5\% over reproduced $π_{0.5}$ while maintaining efficient closed-loop control.
Related papers
- GaussianWAM: Distilling Geometry and Semantics from 3D Gaussian Fields into World-Action Models [21.006803571591707]
World-Action Models (WAMs) jointly learn future visual prediction and action generation, using video dynamics as a representation-learning signal for robotic manipulation.<n>We propose textbfGaussianWAM, a training-time representation-enhancement framework that organizes geometric and semantic supervision through a 3D Gaussian field.
arXiv Detail & Related papers (2026-08-25T15:34:13Z) - SG-WAM: Self-Guided World Modeling in Geometry-Aware Policy Space [12.198811616220835]
World Action Models (WAMs) couple action generation with prediction of future states.<n>We propose SG-WAM, a self-guided framework that learns geometry-aware action-conditioned dynamics directly in the policy-derived representation space.
arXiv Detail & Related papers (2026-08-02T17:25:08Z) - Geometric Action Model for Robot Policy Learning [68.6657929619782]
Generalist robot policies must follow user instructions while reasoning about how objects, cameras, and robot actions interact in the 3D physical world.<n>Recent vision-language-action models (VLAs) and video world-action models (WAMs) inherit strong semantic or temporal priors from large-scale foundation models.<n>We propose the Geometric Action Model (GAM), a language-conditioned manipulation policy.
arXiv Detail & Related papers (2026-06-15T17:58:03Z) - GaussianDream: A Feed-Forward 3D Gaussian World Model for Robotic Manipulation [54.671815855499034]
Vision-language-action (VLA) policies have advanced language-conditioned robotic manipulation by transferring semantic priors to action generation.<n>Standard action-imitation learning often lacks sufficient modeling of explicit 3D spatial information, dense geometric supervision, and future environment evolution.<n>We propose textbfGaussianDream, a feed-forward 3D Gaussian world-model plug-in.
arXiv Detail & Related papers (2026-05-20T05:51:30Z) - Self-Correcting VLA: Online Action Refinement via Sparse World Imagination [55.982504915794514]
We propose Self-Correcting VLA (SC-VLA), which achieve self-improvement by intrinsically guiding action refinement through sparse imagination.<n>SC-VLA achieve state-of-the-art performance, yielding the highest task throughput with 16% fewer steps and a 9% higher success rate than the best-performing baselines.
arXiv Detail & Related papers (2026-02-25T06:58:06Z) - GWM: Towards Scalable Gaussian World Models for Robotic Manipulation [53.51622803589185]
We propose a novel branch of world model named Gaussian World Model (GWM) for robotic manipulation.<n>At its core is a latent Diffusion Transformer (DiT) combined with a 3D variational autoencoder, enabling fine-grained scene-level future state reconstruction.<n>Both simulated and real-world experiments depict that GWM can precisely predict future scenes conditioned on diverse robot actions.
arXiv Detail & Related papers (2025-08-25T02:01:09Z) - FLARE: Robot Learning with Implicit World Modeling [87.81846091038676]
$textbfFLARE$ integrates predictive latent world modeling into robot policy learning.<n>$textbfFLARE$ achieves state-of-the-art performance, outperforming prior policy learning baselines by up to 26%.<n>Our results establish $textbfFLARE$ as a general and scalable approach for combining implicit world modeling with high-frequency robotic control.
arXiv Detail & Related papers (2025-05-21T15:33:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.