Intercepting the Future: Latent-Space Predictive World Model for Dynamic VLA Manipulation

• URL: http://arxiv.org/abs/2606.02486v1
• Date: Mon, 01 Jun 2026 16:55:38 GMT
• Title: Intercepting the Future: Latent-Space Predictive World Model for Dynamic VLA Manipulation
• Abstract summary: AHEAD (Anticipatory Horizon Extrapolation with Adaptive Dynamics) is a predict-then-act wrapper that augments a frozen VLA with a motion-aware latent world model.<n>AHEAD adds 4.9M parameters to a frozen 7B OpenVLA and reaches 79 to 97% success across 20 dynamic simulation scenarios.
• Score: 6.790887409995267
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-Language-Action (VLA) models generalize across static manipulation but fail when objects move during task execution. They map the current observation to an action and assume the scene is stationary between observation and execution, so at any non-trivial object speed the resulting latency exceeds the time available to grasp. We close this gap with AHEAD (Anticipatory Horizon Extrapolation with Adaptive Dynamics), a predict-then-act wrapper that augments a frozen VLA with a motion-aware latent world model. A small world model trained on manipulation video forecasts future patch tokens in the VLA's feature space, conditioned on per-token velocity and acceleration from optical flow. A language-and-motion saliency mask concentrates prediction on task-relevant patches, and the model rolls forward for an adaptive horizon, halting when prediction uncertainty crosses a threshold. The frozen action decoder then receives the predicted future tokens in place of the current ones. AHEAD adds 4.9M parameters to a frozen 7B OpenVLA and reaches 79 to 97% success across 20 dynamic simulation scenarios where the strongest baseline reaches 31 to 58%. On a physical UFactory xArm 7, AHEAD succeeds on 29/30 to 30/30 on three conveyor and rolling-ball tasks, 23/30 on paddle interception, and 19/30 on projectile catching where every baseline scores 0/30.

Related papers

• GaussianDream++: Efficient 3D Gaussian World Modeling for Robotic Manipulation [55.47130289185285]
  Vision-Language-Action (VLA) policies have advanced language-conditioned robotic manipulation.<n>We present textbfMethodname, a compact, policy-native extension that inserts textbfWorld State Tokens and textbfWorld Prediction Tokens directly into the VLA backbone.
  arXiv  Detail & Related papers  (2026-08-26T11:41:55Z)
• Latent Action as Intention Enables Efficient Future Imagination for World Action Models [20.031922437200524]
  We introduce **LAWA**, a WAM architecture that uses compact latent actions as an operational representation of future intentions.<n>On RoboCasa, LAWA achieves state-of-the-art average success rates of 65.6% and 80.8% in the few-shot and full data settings.<n>It also preserves the performance level of the matched Joint-WAM variant while requiring 42.9% lower inference latency.
  arXiv  Detail & Related papers  (2026-08-25T17:59:03Z)
• ForeTime-VLA: Causal Future-Token Distillation from a World Action Model for Conveyor-Belt Manipulation [6.874305530359895]
  We introduce ForeTime-VLA, a dense pi0.5 policy that distills a future-aware, action-equivalent representation from a frozen Fast-WAM-derived teacher.<n>Online, an eight-frame history encoder predicts this target together with manipulation phase and normalized time-to-transition.<n>Test MAE decreases from 0.134119 to 0.130593 (2.63%); paired-bootstrap 95% CI: 0.82-4.48% improvement, and test L2 decreases by 3.02%, at a 2.46-2.93% latency cost.
  arXiv  Detail & Related papers  (2026-08-21T04:40:41Z)
• Foresight Without Seeing: Latent Futures for World Action Models [31.423405133926334]
  We propose ForeWAM, a dynamics-conditioned direct-policy WAM that provides predictive context for action generation without decoding future videos.<n>We also introduce dynamics supervised by a frozen latent action teacher, encouraging the implicit future states to capture interaction-induced transitions such as object motion, contact changes, and task progress.<n>Without embodied robot data pretraining, the standard and accelerated variants of ForeWAM achieve average success rates of 96.7% and 96.9% on LIBERO, respectively.
  arXiv  Detail & Related papers  (2026-08-12T03:27:43Z)
• LocalNav: Distilling Frontier VLMs and Embodied RL for On-Device Object Goal Navigation [53.88286427894783]
  Vision Language Models (VLMs) have emerged in the robotic domain as a powerful tool that enables environmental perception with language context.<n>Yet, their computational footprint typically confines them to cloud execution, hindering low-latency inference with local deployment on resource-constrained robots.<n>We present a distillation strategy that transfers complex spatial-semantic reasoning from large frontier models into a lightweight, 4B- parameter local VLM for edge execution on embedded GPU devices.
  arXiv  Detail & Related papers  (2026-06-26T09:11:59Z)
• LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies [31.88655699486955]
  We present LaWAM, a latent-action-conditioned Latent World Model (LaWM)<n>LaWAM exposes predictive dynamics to robot policies through compact latent visual subgoals instead of reconstructed future video.<n>LaWAM runs in 187 ms per action-chunk prediction and achieves up to 24x lower wall-clock latency than pixel-space WAMs.
  arXiv  Detail & Related papers  (2026-06-14T12:06:58Z)
• OASIS: Observation-Action Space Alignment via SE(3) Trajectory Prediction for Robotic Manipulation [41.754765514902736]
  We propose a visuomotor policy that aligns the intermediate representation with the action space via $SE(3)$ trajectory prediction.<n>OASIS couples a 3D-aware feature encoder that fuses vision-language and metric-depth features with an $SE(3)$ trajectory predictor.<n>OASIS outperforms VLA and WAM baselines in success rate and out-of-distribution generalization.
  arXiv  Detail & Related papers  (2026-05-25T13:28:33Z)
• GaussianDream: A Feed-Forward 3D Gaussian World Model for Robotic Manipulation [54.671815855499034]
  Vision-language-action (VLA) policies have advanced language-conditioned robotic manipulation by transferring semantic priors to action generation.<n>Standard action-imitation learning often lacks sufficient modeling of explicit 3D spatial information, dense geometric supervision, and future environment evolution.<n>We propose textbfGaussianDream, a feed-forward 3D Gaussian world-model plug-in.
  arXiv  Detail & Related papers  (2026-05-20T05:51:30Z)
• GEM: Gaussian Evolution Model for Occupancy Forecasting and Motion Planning [13.877624821024535]
  We propose a non-autoregressive occupancy world model for future semantic occupancy forecasting and motion planning.<n>We show that GEM achieves state-of-the-art future semantic occupancy forecasting and strong motion planning performance.
  arXiv  Detail & Related papers  (2026-05-17T22:52:53Z)
• From Imagined Futures to Executable Actions: Mixture of Latent Actions for Robot Manipulation [88.39072412680633]
  We propose MoLA, a control-oriented interface that transforms imagined future videos into executable representations.<n>We evaluate our approach on simulated benchmarks and real-world robot manipulation tasks.
  arXiv  Detail & Related papers  (2026-05-12T14:15:16Z)
• When to Trust Imagination: Adaptive Action Execution for World Action Models [42.51856318901667]
  World Action Models (WAMs) have recently emerged as a promising paradigm for robotic manipulation by jointly predicting future visual observations and future actions.<n>Current WAMs execute a fixed number of predicted actions after each model inference, leaving the robot blind to whether the imagined future remains consistent with the actual physical rollout.<n>We formulate adaptive WAM execution as a future-reality verification problem: the robot should execute longer when the WAM-predicted future remains reliable, and replan earlier when reality deviates from imagination.
  arXiv  Detail & Related papers  (2026-05-07T13:18:28Z)
• STARRY: Spatial-Temporal Action-Centric World Modeling for Robotic Manipulation [30.24537271257149]
  STARRY is a world-model-enhanced action-generation policy that aligns spatial-temporal prediction and action generation.<n>On RoboTwin 2.0, STARRY achieves 93.82% / 93.30% average success under Clean and Randomized settings.<n>These results demonstrate the effectiveness of action-centric spatial-temporal world modeling for spatially and temporally demanding robotic manipulation.
  arXiv  Detail & Related papers  (2026-04-29T16:13:39Z)
• OFlow: Injecting Object-Aware Temporal Flow Matching for Robust Robotic Manipulation [55.56490813640669]
  VLA models typically act only on the current frame, while future prediction and object-aware reasoning are often learned in separate latent spaces.<n>We propose OFlow, a framework that addresses both limitations by unifying temporal foresight and object-aware reasoning in a shared semantic latent space.<n>Our method forecasts future latents with temporal flow matching, factorizes them into object-aware representations that emphasize physically relevant cues.
  arXiv  Detail & Related papers  (2026-04-20T06:38:01Z)
• LatentPilot: Scene-Aware Vision-and-Language Navigation by Dreaming Ahead with Latent Visual Reasoning [51.969318585152116]
  LatentPilot exploits future observations during training as a valuable data source to learn action-conditioned visual dynamics.<n>We propose a flywheel-style training mechanism that iteratively collects on-policy trajectories and retrains the model to better match the agent's behavior distribution.<n> Experiments on R2R-CE, RxR-CE, and R2R-PE benchmarks achieve new SOTA results, and real-robot tests across diverse environments demonstrate LatentPilot's superior understanding of environment-action dynamics in scene.
  arXiv  Detail & Related papers  (2026-03-31T02:21:59Z)
• ProgressVLA: Progress-Guided Diffusion Policy for Vision-Language Robotic Manipulation [57.07494675832939]
  Most existing vision-language-action (VLA) models for robotic manipulation lack progress awareness.<n>In this work, we investigate the estimation and integration of task progress, proposing a novel model named textbf vla<n>Experiments on the CALVIN and LIBERO benchmarks, alongside real-world robot deployment, consistently demonstrate substantial improvements in success rates.
  arXiv  Detail & Related papers  (2026-03-29T12:38:11Z)
• Self-Correcting VLA: Online Action Refinement via Sparse World Imagination [55.982504915794514]
  We propose Self-Correcting VLA (SC-VLA), which achieve self-improvement by intrinsically guiding action refinement through sparse imagination.<n>SC-VLA achieve state-of-the-art performance, yielding the highest task throughput with 16% fewer steps and a 9% higher success rate than the best-performing baselines.
  arXiv  Detail & Related papers  (2026-02-25T06:58:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.