Hydra-0: Action Flow for Generalist World Modeling and Control

• URL: http://arxiv.org/abs/2608.18077v1
• Date: Tue, 18 Aug 2026 17:59:30 GMT
• Title: Hydra-0: Action Flow for Generalist World Modeling and Control
• Abstract summary: Hydra-0 is a world model conditioned on action flow, which represents robot actions as pixel motion.<n>Our best configuration achieves 90.4% lower robot-motion error and 60.2% lower object-motion error.<n>A trained action head maps the resulting latent features to executable actions without requiring task-specific expert robot demonstrations.
• Score: 64.88849745875753
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce Hydra-0, a generalist world model conditioned on action flow, which represents robot actions as pixel motion. This shared visual interface enables generalist world modeling and control by learning action consequences across embodiments, tasks, environments, and video-generation backbones. Our best configuration achieves 90.4% lower robot-motion error and 60.2% lower object-motion error than our action-conditioned baseline, while supporting zero-shot composition and data-efficient adaptation. On the RoboLab benchmark, Hydra-0 achieves a Pearson correlation of r=0.96 between replayed and reference success rates. Finally, we uncover an emergent inverse mode of this interface: a world action model that predicts compatible robot motion from desired object flow transferred from a human demonstration. A trained action head maps the resulting latent features to executable actions without requiring task-specific expert robot demonstrations. Together, these results demonstrate the potential of action flow as a shared control interface connecting heterogeneous training data, open-loop policy evaluation, and robot control.

Related papers

• AnyWorld: Factorized Egocentric World Models for Cross-Embodiment Generalization [88.25615187835321]
  We propose AnyWorld, a cross-embodiment world modeling framework.<n>Our model factorizes an interaction into action, camera, and embodiment.<n>We train the model with large-scale human interaction pretraining.
  arXiv  Detail & Related papers  (2026-08-29T12:55:15Z)
• Masked Visual Actions for Unified World Modeling [96.12988421978463]
  We introduce Masked Visual Actions, a pixel-space control interface that expresses action as a partially revealed trajectory of an arbitrary entity in a video.<n>Revealing robot motion makes the model act as a forward dynamics model that predicts the scene's response to low-level robot actions, while revealing desired object motion makes the same model recover robot behavior consistent with that outcome.
  arXiv  Detail & Related papers  (2026-07-21T17:59:11Z)
• WALA Learning Executable Latent Actions from Action-Labeled Demonstrations and Action-Free Videos [38.959415159248046]
  WALA is a framework for learning executable latent actions from both action-labeled demonstrations and action-free videos.<n>We show that WALA achieves strong performance on RoboTwin, sets a new state-of-the-art result on RoboCasa with 75.2% average success.
  arXiv  Detail & Related papers  (2026-07-13T11:02:04Z)
• ABot-M0.5: Unified Mobility-and-Manipulation World Action Model [27.262271208923995]
  ABot-M0.5 is built on the insight that mobile manipulation requires alignment at three levels: temporal granularity, action space, and train-test consistency.<n> Experiments on challenging mobile and fine-grained manipulation benchmarks demonstrate that ABot-M0.5 achieves state-of-the-art performance in both long-horizon task success and finegrained control accuracy.
  arXiv  Detail & Related papers  (2026-07-01T09:21:20Z)
• DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos [110.98100817695307]
  We introduce DreamDojo, a foundation world model that learns diverse interactions and dexterous controls from 44k hours of egocentric human videos.<n>Our work enables several important applications based on generative world models, including live teleoperation, policy evaluation, and model-based planning.
  arXiv  Detail & Related papers  (2026-02-06T18:49:43Z)
• MiVLA: Towards Generalizable Vision-Language-Action Model with Human-Robot Mutual Imitation Pre-training [102.850162490626]
  We propose MiVLA, a vision-language-action model empowered by human-robot mutual imitation pre-training.<n>We show that MiVLA achieves strong improved generalization capability, outperforming state-of-the-art VLAs.
  arXiv  Detail & Related papers  (2025-12-17T12:59:41Z)
• ViPRA: Video Prediction for Robot Actions [33.310474967770894]
  We present Video Prediction for Robot Actions (ViPRA), a framework that learns continuous robot control from actionless videos.<n>Instead of directly predicting actions, we train a video-language model to predict both future visual observations and motion-centric latent actions.<n>For downstream control, we introduce a chunked flow matching decoder that maps latent actions to robot-specific continuous action sequences.
  arXiv  Detail & Related papers  (2025-11-11T01:33:03Z)
• DemoHLM: From One Demonstration to Generalizable Humanoid Loco-Manipulation [29.519071338337685]
  We present DemoHLM, a framework for humanoid loco-manipulation on a real humanoid robot from a single demonstration in simulation.<n>whole-body controller maps whole-body motion commands to joint torques and provides omnidirectional mobility for the humanoid robot.<n> Experiments show a positive correlation between the amount of synthetic data and policy performance.
  arXiv  Detail & Related papers  (2025-10-13T10:49:40Z)
• DreamGen: Unlocking Generalization in Robot Learning through Video World Models [120.25799361925387]
  DreamGen is a pipeline for training robot policies that generalize across behaviors and environments through neural trajectories.<n>Our work establishes a promising new axis for scaling robot learning well beyond manual data collection.
  arXiv  Detail & Related papers  (2025-05-19T04:55:39Z)
• Action Flow Matching for Continual Robot Learning [54.10050120844738]
  Continual learning in robotics seeks systems that can constantly adapt to changing environments and tasks.<n>We introduce a generative framework leveraging flow matching for online robot dynamics model alignment.<n>We find that by transforming the actions themselves rather than exploring with a misaligned model, the robot collects informative data more efficiently.
  arXiv  Detail & Related papers  (2025-04-25T16:26:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.