SG-WAM: Self-Guided World Modeling in Geometry-Aware Policy Space

• URL: http://arxiv.org/abs/2608.01397v1
• Date: Sun, 02 Aug 2026 17:25:08 GMT
• Title: SG-WAM: Self-Guided World Modeling in Geometry-Aware Policy Space
• Abstract summary: World Action Models (WAMs) couple action generation with prediction of future states.<n>We propose SG-WAM, a self-guided framework that learns geometry-aware action-conditioned dynamics directly in the policy-derived representation space.
• Score: 12.198811616220835
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: World Action Models (WAMs) couple action generation with prediction of future states. Their effectiveness depends on whether future dynamics are modeled in a space that is both aligned with action generation and sufficiently geometry-aware to capture where and how actions change the scene. Existing WAMs typically satisfy only part of this requirement, relying on either perceptually heavy observation-space targets or auxiliary latent spaces that are not jointly structured for action relevance and geometry. We propose SG-WAM, a self-guided framework that learns geometry-aware action-conditioned dynamics directly in the policy-derived representation space. SG-WAM introduces learnable dynamics tokens and a Self-Guided World Predictor that forecasts their future latent states conditioned on intervening robot actions. Prediction targets are generated by an exponential moving average copy of the same policy backbone, providing stable supervision within the representation family used by the action expert. Geometric supervision further structures the policy image-token representations, providing spatially grounded context for the dynamics tokens and yielding a future-alignment space that is both action-relevant and geometry-aware. Latent future prediction, geometric grounding, and flow-matching action generation are jointly optimized end-to-end in a unified framework. Built on a 0.9B model without large-scale embodied pretraining, SG-WAM achieves 98.5% average success on LIBERO and 73% on LIBERO-Plus, while outperforming strong baselines in both in-distribution and out-of-distribution real-world evaluations.

Related papers

• GaussianDream++: Efficient 3D Gaussian World Modeling for Robotic Manipulation [55.47130289185285]
  Vision-Language-Action (VLA) policies have advanced language-conditioned robotic manipulation.<n>We present textbfMethodname, a compact, policy-native extension that inserts textbfWorld State Tokens and textbfWorld Prediction Tokens directly into the VLA backbone.
  arXiv  Detail & Related papers  (2026-08-26T11:41:55Z)
• GeoWAM: Visual Geometry World Action Models for Autonomous Driving [14.05425826273513]
  We introduce textbfGeoWAM, a visual geometry world action model for autonomous driving.<n>Rather than predicting future images, GeoWAM is pretrained to forecast future scene geometry.<n>A geometry-conditioned action head then leverages these learned geometric dynamics to predict future ego trajectories.
  arXiv  Detail & Related papers  (2026-08-24T16:49:53Z)
• JEPA-WAM: Learning Vision-Language-Action Policies with Joint-Embedding World Modeling [23.76460420882974]
  We introduce JEPA-WAM, a latent WAM built in a pretrained V-JEPA space.<n>It couples latent transition prediction with continuous action generation through a shared predictor.<n>On LIBERO-Plus, JEPA-WAM achieves 79.2%, the best result without large-scale robot-policy pretraining.
  arXiv  Detail & Related papers  (2026-08-10T09:57:54Z)
• Geometric Action Model for Robot Policy Learning [68.6657929619782]
  Generalist robot policies must follow user instructions while reasoning about how objects, cameras, and robot actions interact in the 3D physical world.<n>Recent vision-language-action models (VLAs) and video world-action models (WAMs) inherit strong semantic or temporal priors from large-scale foundation models.<n>We propose the Geometric Action Model (GAM), a language-conditioned manipulation policy.
  arXiv  Detail & Related papers  (2026-06-15T17:58:03Z)
• GeoSem-WAM: Geometry- and Semantic-Aware World Action Models [13.18627268862803]
  We propose a structured world modeling framework that enhances latent representations through geometric and semantic supervision.<n>Our approach preserves efficient inference by avoiding explicit future rollout or video generation at test time.
  arXiv  Detail & Related papers  (2026-06-02T05:48:02Z)
• Beyond Point-Wise Matching: Structural Representation Alignment for Accelerating Diffusion Transformers [93.3976834364707]
  We propose sREPA, a structural REPresentation Alignment framework to enforce consistency in the relational geometry of feature maps.<n>By encouraging the model to internalize holistic spatial layouts and structural correlations from pre-trained features, sREPA achieves faster and more stable convergence.
  arXiv  Detail & Related papers  (2026-05-16T12:01:04Z)
• World Action Models: The Next Frontier in Embodied AI [123.5787299299832]
  Vision-Language-Action (VLA) models have achieved strong semantic generalization for embodied policy learning.<n>They learn reactive observation-to-action mappings without explicitly modeling how the physical world evolves under intervention.<n>A growing body of work addresses this limitation by integrating world models, predictive models of environment dynamics, into the action generation pipeline.
  arXiv  Detail & Related papers  (2026-05-12T13:10:52Z)
• HarmoWAM: Harmonizing Generalizable and Precise Manipulation via Adaptive World Action Models [58.191567345416836]
  World Action Models (WAMs) have emerged as a promising paradigm for robot control by modeling physical dynamics.<n>HarmoWAM fully leverages a world model to unify predictive and reactive control, enabling both general transit and precise manipulation.<n>We construct three training-unseen test environments across six real-world robotic tasks, covering variations in background, position, and object semantics.
  arXiv  Detail & Related papers  (2026-05-11T17:59:56Z)
• STARRY: Spatial-Temporal Action-Centric World Modeling for Robotic Manipulation [30.24537271257149]
  STARRY is a world-model-enhanced action-generation policy that aligns spatial-temporal prediction and action generation.<n>On RoboTwin 2.0, STARRY achieves 93.82% / 93.30% average success under Clean and Randomized settings.<n>These results demonstrate the effectiveness of action-centric spatial-temporal world modeling for spatially and temporally demanding robotic manipulation.
  arXiv  Detail & Related papers  (2026-04-29T16:13:39Z)
• Self-Correcting VLA: Online Action Refinement via Sparse World Imagination [55.982504915794514]
  We propose Self-Correcting VLA (SC-VLA), which achieve self-improvement by intrinsically guiding action refinement through sparse imagination.<n>SC-VLA achieve state-of-the-art performance, yielding the highest task throughput with 16% fewer steps and a 9% higher success rate than the best-performing baselines.
  arXiv  Detail & Related papers  (2026-02-25T06:58:06Z)
• LaDi-WM: A Latent Diffusion-based World Model for Predictive Manipulation [45.02469804709771]
  We propose LaDi-WM, a world model that predicts the latent space of future states using diffusion modeling.<n>We show that LaDi-WM significantly enhances policy performance by 27.9% on the LIBERO-LONG benchmark and 20% on the real-world scenario.
  arXiv  Detail & Related papers  (2025-05-13T04:42:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.