Learning 4D Geometric Priors for Inference-Efficient World Action Models

• URL: http://arxiv.org/abs/2607.05468v1
• Date: Mon, 06 Jul 2026 07:10:22 GMT
• Title: Learning 4D Geometric Priors for Inference-Efficient World Action Models
• Abstract summary: World Action Models (WAMs) have shown strong potential for robotic manipulation by jointly modeling visual future dynamics and executable action sequences.<n>We propose MECo-WAM, a Multi-Expert Co-Training World Action Model that injects action-relevant 4D geometric priors into video-action representations.<n>Experiments on LIBERO (98.2%), RoboTwin 2.0 (92.6%), and challenging real-world manipulation tasks show that MECo-WAM improves manipulation performance without increasing inference cost.
• Score: 26.63633606928882
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: World Action Models (WAMs) have shown strong potential for robotic manipulation by jointly modeling visual future dynamics and executable action sequences. However, existing video-action co-training methods primarily optimize appearance-oriented video latents, which may insufficiently capture the temporally evolving geometry required for precise manipulation. We propose MECo-WAM, a Multi-Expert Co-Training World Action Model that injects action-relevant 4D geometric priors into video-action representations while preserving the original lightweight inference graph. During training, MECo-WAM combines video and action experts with a lightweight 4D expert supervised by relational targets from a frozen VGGT encoder. Asymmetric expert visibility prevents non-causal shortcuts from auxiliary geometry to action generation. To transfer geometric knowledge into the deployed video-action pathway, we introduce decayed 4D read-mask attention, which provides restricted current-frame geometric guidance early in training and progressively removes this dependency. We further propose action-aware temporal geometric distillation, which aligns within-frame geometric relations and their temporal evolution while emphasizing visual regions most relevant to robot actions. At deployment, all auxiliary 4D components are removed. Experiments on LIBERO (98.2%), RoboTwin 2.0 (92.6%), and challenging real-world manipulation tasks show that MECo-WAM improves manipulation performance without increasing inference cost.

Related papers

• GeoLAM: Learning Geometry-Grounded Latent Actions from Unlabeled Human Videos [8.414561051327915]
  We present GeoLAM, a framework for learning geometry-grounded latent actions from action-free human videos.<n>GeoLAM combines future-frame reconstruction through a frozen geometric feature hierarchy with motion supervision from a training-only 4D geometry teacher.
  arXiv  Detail & Related papers  (2026-09-15T12:31:34Z)
• LAWM-3D: Learning 3D-Aware Latent Actions from Human Videos for Generalizable Robot World Models [64.11117186468533]
  World models enable agents to perform forward rollout and planning without real-world interaction.<n>Most existing LAMs rely on single-view inputs and operate primarily in 2D pixel space.<n>We propose LAWM-3D, which introduces three tightly coupled key designs for learning 3D-aware latent actions.
  arXiv  Detail & Related papers  (2026-08-06T07:47:01Z)
• WAM4D: Fast 4D World Action Model via Spatial Register Tokens [52.16661403052964]
  We present WAM4D, a fast 4D world action model that uses lightweight spatial register tokens as training-time future-depth readouts.<n>We show that WAM4D improves spatial consistency and achieves competitive action prediction while maintaining efficient inference.
  arXiv  Detail & Related papers  (2026-06-12T02:49:34Z)
• GEM-4D: Geometry-Enhanced Video World Models for Robot Manipulation [72.52773248997929]
  Video world models can generate realistic futures from a single instruction, but they often fail to preserve consistent point-level motion over time.<n>We present GEM-4D, a geometry-grounded video world model that injects dense 4D correspondence supervision into the video generative backbone during training.<n>In inverse dynamics module converts correspondence-consistent video rollouts into executable robot trajectories, enabling direct deployment in both real-world and simulated manipulation.
  arXiv  Detail & Related papers  (2026-05-20T21:36:44Z)
• MVISTA-4D: View-Consistent 4D World Model with Test-Time Action Inference for Robotic Manipulation [27.70398018267795]
  This work proposes a novel embodied 4D world model that enables geometrically consistent, arbitrary-view RGBD generation.<n>Experiments on three datasets demonstrate strong performance on both 4D scene generation and downstream manipulation.
  arXiv  Detail & Related papers  (2026-02-10T15:19:17Z)
• Diff4Splat: Controllable 4D Scene Generation with Latent Dynamic Reconstruction Models [79.06910348413861]
  We introduce Diff4Splat, a feed-forward method that synthesizes controllable and explicit 4D scenes from a single image.<n>Given a single input image, a camera trajectory, and an optional text prompt, Diff4Splat directly predicts a deformable 3D Gaussian field that encodes appearance, geometry, and motion.
  arXiv  Detail & Related papers  (2025-11-01T11:16:25Z)
• DynaRend: Learning 3D Dynamics via Masked Future Rendering for Robotic Manipulation [52.136378691610524]
  We present DynaRend, a representation learning framework that learns 3D-aware and dynamics-informed triplane features.<n>By pretraining on multi-view RGB-D video data, DynaRend jointly captures spatial geometry, future dynamics, and task semantics in a unified triplane representation.<n>We evaluate DynaRend on two challenging benchmarks, RLBench and Colosseum, demonstrating substantial improvements in policy success rate, generalization to environmental perturbations, and real-world applicability across diverse manipulation tasks.
  arXiv  Detail & Related papers  (2025-10-28T10:17:11Z)
• 4D Driving Scene Generation With Stereo Forcing [62.47705572424127]
  Current generative models struggle to synthesize dynamic 4D driving scenes that simultaneously support temporal extrapolation and spatial novel view synthesis (NVS) without per-scene optimization.<n>We present PhiGenesis, a unified framework for 4D scene generation that extends video generation techniques with geometric and temporal consistency.
  arXiv  Detail & Related papers  (2025-09-24T15:37:17Z)
• GAF: Gaussian Action Field as a 4D Representation for Dynamic World Modeling in Robotic Manipulation [47.471097712217386]
  Existing approaches typically follow either a Vision-to-Action (V-A) paradigm, predicting actions directly from visual inputs, or a Vision-to-3D-to-Action (V-3D-A) paradigm, leveraging intermediate 3D representations.<n>In this paper, we adopt a V-4D-A framework that enables direct action reasoning from motion-aware 4D representations via a Gaussian Action Field (GAF)<n>Experiments demonstrate significant improvements, with achieving +11.5385 dB PSNR, +0.3864 SSIM and -0.5574 LPIPS improvements in reconstruction quality, while boosting the average +7.3% success
  arXiv  Detail & Related papers  (2025-06-17T02:55:20Z)
• Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstruction [72.54905331756076]
  We introduce Geo4D, a method to repurpose video diffusion models for monocular 3D reconstruction of dynamic scenes.<n>By leveraging the strong dynamic priors captured by large-scale pre-trained video models, Geo4D can be trained using only synthetic data.<n>We propose a new multi-modal alignment algorithm to align and fuse these modalities, as well as a sliding window approach at inference time.
  arXiv  Detail & Related papers  (2025-04-10T17:59:55Z)
• PartRM: Modeling Part-Level Dynamics with Large Cross-State Reconstruction Model [23.768571323272152]
  PartRM is a novel 4D reconstruction framework that simultaneously models appearance, geometry, and part-level motion from multi-view images of a static object.<n>We introduce the PartDrag-4D dataset, providing multi-view observations of part-level dynamics across over 20,000 states.<n> Experimental results show that PartRM establishes a new state-of-the-art in part-level motion learning and can be applied in manipulation tasks in robotics.
  arXiv  Detail & Related papers  (2025-03-25T17:59:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.