$ω$-EVA: Envision, Verify, and Act with Latent Interactive World Models

• URL: http://arxiv.org/abs/2606.09457v1
• Date: Mon, 08 Jun 2026 13:12:56 GMT
• Title: $ω$-EVA: Envision, Verify, and Act with Latent Interactive World Models
• Abstract summary: Embodied policies typically map current observations directly to actions, leaving candidate-action consequences implicit.<n>We introduce $$-EVA, a latent interactive world model that realizes an Envision-Verify-Act loop for embodied action generation.<n>Its three-stage framework learns action-conditioned latent dynamics, trains a language-conditioned flow policy, and feeds the policy's proposal back through the world model.
• Score: 28.544954130906135
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Embodied policies typically map current observations directly to actions, leaving candidate-action consequences implicit. World models provide predictive supervision, representations, or external simulation, but rarely let a policy inspect the imagined consequence of its own proposal before acting. We introduce $ω$-EVA, a latent interactive world model that realizes an Envision--Verify--Act loop for embodied action generation. Its three-stage framework learns action-conditioned latent dynamics, trains a language-conditioned flow policy on dynamics-aware visual representations, and feeds the policy's proposal back through the world model. A tri-branch refiner jointly reasons over the current state, proposal-conditioned future, and proposed action to produce the final action chunk. Because consequence reasoning remains in latent feature space, $ω$-EVA avoids generating future videos at inference. Evaluations across diverse single-arm, bimanual, long-horizon, and perturbed simulation settings show that the complete interaction pipeline consistently improves the proposal policy, while latent diagnostics indicate meaningful action-conditioned future structure. With approximately 1.2B parameters and no additional robot-data pretraining, $ω$-EVA demonstrates a compact and competitive performance--scale--data trade-off, making the world model an active action-feedback module rather than a passive predictor.

Related papers

• JEPA Policy: Diffusion-Free Imitation Learning via Paired Action and Future Representation Prediction [6.889062419237111]
  JEPA Policy is a diffusion-free framework that uses the action chunk and its observed future representation as paired training targets.<n>Future prediction can therefore shape the representation used to generate actions.<n>Results support paired future-representation supervision as a practical approach to low-latency visuomotor imitation without iterative generative sampling.
  arXiv  Detail & Related papers  (2026-09-09T02:42:20Z)
• Beyond Instance Slots: Semantically Rich World Models for Physical Interaction Planning [3.284163275977256]
  We present the Semantically Rich World Model (SR-WM), a task-conditioned world model structured around five functional roles.<n>Within SR-WM, a visual entity encoder extracts soft entity hypotheses from pretrained patch features, allowing segmentation masks to serve as optional proposal priors.<n>A role binder maps these hypotheses to task-specific roles, while an action conditioned dynamics model predicts role transitions alongside fine-grained semantics.
  arXiv  Detail & Related papers  (2026-08-23T08:37:59Z)
• SLIM-0.5B: Learning Action-Grounded Predictive Latents for Robot Manipulation [35.54319311306262]
  We present SLIM, a compact latent interaction policy.<n>It learns representations through self-supervised masked trajectory prediction.<n>SLIM matches or exceeds representative large-scale VLA and world-action-model baselines with fewer parameters.
  arXiv  Detail & Related papers  (2026-08-10T15:58:39Z)
• World Action Models: A Survey [100.95337034529263]
  World Action Models (WAMs) are embodied predictive-action models that make a forecast of the future available to action.<n>Recent WAMs repurpose large video generation models, and a parallel line relies on language or vision-language backbones without a video-generation core.<n>This survey gives the field a common account. It first clarifies these boundaries, then organizes existing works through two complementary views.
  arXiv  Detail & Related papers  (2026-06-18T17:05:19Z)
• SC3-Eval: Evaluating Robot Foundation Models via Self-Consistent Video Generation [73.2142090645987]
  SC3-Eval is a self-consistent video generation recipe that adapts a pre-trained video foundation model into an accurate policy evaluator.<n>We demonstrate SC3-Eval rollouts reproduce the failure modes that policies exhibit in real-world rollouts, supporting fine-grained diagnostic comparison.
  arXiv  Detail & Related papers  (2026-06-17T02:15:46Z)
• Discrete-WAM: Unified Discrete Vision-Action Token Editing for World-Policy Learning [28.162567330430928]
  We introduce Discrete-WAM, a unified vision-action world-policy framework that represents visual observations, future states, high-level decisions, and ego actions within a shared token space.<n>Built on this discrete alignment, Discrete-WAM jointly trains world modeling, world-policy modeling, and policy modeling through multi-task and multi-stage pretraining.<n>Experiments show that Discrete-WAM achieves strong planning performance while supporting controllable future generation, counterfactual evaluation, surprise-based world-model analysis, and efficient parallel policy decoding.
  arXiv  Detail & Related papers  (2026-06-04T03:16:31Z)
• From Imagined Futures to Executable Actions: Mixture of Latent Actions for Robot Manipulation [88.39072412680633]
  We propose MoLA, a control-oriented interface that transforms imagined future videos into executable representations.<n>We evaluate our approach on simulated benchmarks and real-world robot manipulation tasks.
  arXiv  Detail & Related papers  (2026-05-12T14:15:16Z)
• AGWM: Affordance-Grounded World Models for Environments with Compositional Prerequisites [18.635313325437664]
  We propose AGWM (Affordance-Grounded World Model), which learns an abstract affordance structure represented as a DAG of prerequisite dependencies to explicitly track the dynamic executability of actions.<n> Experiments on game-based simulated environments demonstrate the effectiveness of our method by achieving lower multi-step prediction error, better generalization to novel configurations, and improved interpretability.
  arXiv  Detail & Related papers  (2026-05-07T18:46:44Z)
• Being-H0.7: A Latent World-Action Model from Egocentric Videos [32.77431338471086]
  We present Being-H0.7, a latent world-action model that brings future-aware reasoning into VLA-style policies.<n>Being-H0.7 inserts learnable latent queries between perception and action as a compact reasoning interface.
  arXiv  Detail & Related papers  (2026-04-30T14:16:15Z)
• World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry [82.93104394404781]
  General-purpose world models promise scalable policy evaluation, optimization, and planning.<n>We propose World Action Verifier (WAV), a framework that enables world models to identify their own prediction errors and self-improve.
  arXiv  Detail & Related papers  (2026-04-02T12:48:36Z)
• Ego-centric Predictive Model Conditioned on Hand Trajectories [52.531681772560724]
  In egocentric scenarios, anticipating both the next action and its visual outcome is essential for understanding human-object interactions.<n>We propose a unified two-stage predictive framework that jointly models action and visual future in egocentric scenarios.<n>Our approach is the first unified model designed to handle both egocentric human activity understanding and robotic manipulation tasks.
  arXiv  Detail & Related papers  (2025-08-27T13:09:55Z)
• WorldGym: World Model as An Environment for Policy Evaluation [41.204900701616914]
  WorldGym is an autoregressive, action-conditioned video generation model which serves as a proxy to real world environments.<n> Policies are evaluated via Monte Carlo rollouts in the world model, with a vision-language model providing rewards.<n>We show that WorldGym is able to preserve relative policy rankings across different policy versions, sizes, and training checkpoints.
  arXiv  Detail & Related papers  (2025-05-31T15:51:56Z)
• Deep Active Inference Agents for Delayed and Long-Horizon Environments [1.693200946453174]
  AIF agents rely on accurate immediate predictions and exhaustive planning, a limitation that is exacerbated in delayed environments.<n>We propose a generative-policy architecture featuring a multi-step latent transition that lets the generative model predict an entire horizon in a single look-ahead.<n>We evaluate our agent in an environment that mimics a realistic industrial scenario with delayed and long-horizon settings.
  arXiv  Detail & Related papers  (2025-05-26T11:50:22Z)
• Instance-Aware Predictive Navigation in Multi-Agent Environments [93.15055834395304]
  We propose an Instance-Aware Predictive Control (IPC) approach, which forecasts interactions between agents as well as future scene structures. We adopt a novel multi-instance event prediction module to estimate the possible interaction among agents in the ego-centric view. We design a sequential action sampling strategy to better leverage predicted states on both scene-level and instance-level.
  arXiv  Detail & Related papers  (2021-01-14T22:21:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.