Fugu-MT: arxivの論文翻訳

Summary

This week's theme centers on using embodied world models not just to generate realistic futures, but to evaluate, supervise, and improve robotic policies. Representative papers argue that action-conditioned prediction from large-scale interaction data can yield better physical reasoning, stronger transferable representations, and more data-efficient vision-language-action learning through synthetic data and model-based reinforcement learning.

Situation

The representative introductions frame a common limitation in current robotic learning: strong vision-language-action systems still depend heavily on expensive real-world interaction data and often act myopically, while passive video models can look realistic without capturing the causal structure needed for physical reasoning. In response, these papers position embodied world models as predictive engines trained on action-outcome data, so that models learn how the world changes under intervention rather than only how it appears.

Within that framing, the emphasis shifts from pure generation to evaluation and representation use. WoW argues for benchmarks that test physical consistency, causal reasoning, and executable action grounding, and integrates self-supervised visual features into diffusion-based world models to strengthen perceptual representations. The GigaBrain papers extend this logic by using world-model-generated trajectories to expand training diversity and by conditioning policies on predicted futures and values, treating the world model as a source of foresight, supervision, and iterative policy improvement rather than a standalone simulator.

Infographic (English)

Embodied World Models for Robot Learning situation infographic

Progress

RynnBrain 1.1: Towards More Capable and Generalizable Embodied Foundation Model <See Details on Fugu-MT>

RynnBrain 1.1: Towards More Capable and Generalizable Embodied Foundation Model: RynnBrain 1.1は2B,9B,122B-A10Bスケールにまたがる基礎モデルである。 RynnBrain 1.1は、具体的知覚、空間的推論の局在、計画をサポートする。 また,RynnBrain-VLAを統一的なクロスボデーメントアクション空間とエンボデーメント固有のマスキングで開発し,Unitree G1,Astribot-S1,Tianji-Wujiに展開する。 It connects to Model Evaluation / Representation Learning through the paper's concrete task, method, evidence, or application setting.

Agentic Real2Sim: Physics-based World Modeling with Vision-Language Agents <See Details on Fugu-MT>

Agentic Real2Sim uses vision-language agents to automatically convert recorded robot interactions into physics-based simulatable digital twins across rigid, deformable, and humanoid settings. Compared with manual or partial Real2Sim pipelines, it provides an end-to-end agentic framework that produces reality-aligned twins explicitly designed for downstream policy learning and evaluation.

RoboInter1.5: A Holistic Intermediate Representation Suite for Embodied World Modeling and Robotic Manipulation <See Details on Fugu-MT>

RoboInter1.5 provides a unified data, benchmark, and model suite for manipulation-oriented intermediate representations in embodied world modeling. Compared with earlier work focused on raw world-model outputs, it adds an explicit intermediate representation layer for organizing training and evaluation around manipulation structure.

Masked Visual Actions for Unified World Modeling <See Details on Fugu-MT>

Masked Visual Actions encodes actions as partially revealed visual trajectories, enabling a single pretrained video model to serve as both a forward dynamics model and an inverse model for action recovery. Unlike prior action-conditioned video prediction with embodiment-specific inputs, it introduces a pixel-space control interface that unifies forward and inverse reasoning within the model's native visual representation.

KineBench: Benchmarking Embodied World Models via IDM-Free Kinematic Grounding <See Details on Fugu-MT>

KineBench introduces a closed-loop benchmark for embodied world models that evaluates physical consistency without relying on inverse dynamics models for action extraction. Compared with prior benchmarks that depend on separately trained IDMs, it provides IDM-free kinematic grounding as a more direct measure of physical plausibility.

PAVXploreRL: Physical-Action-Visual World Model Reinforcement Learning with Action Exploration <See Details on Fugu-MT>

PAVXploreRL builds a reinforcement learning framework with action exploration on top of a pretrained latent world model for scalable policy evaluation. Compared with using world models mainly for prediction or synthetic data generation, it demonstrates direct policy improvement through world-model-based exploration.

ABot-World-0: Infinite Interactive World Rollout on a Single Desktop GPU <See Details on Fugu-MT>

ABot-World-0 enables real-time, long-horizon closed-loop interactive rollouts from an action-conditioned video world model on a single desktop GPU, with an agent-driven data collection pipeline. Compared with offline or short-horizon predictive setups, it adds scalable rollout with synchronized action-text annotation and deterministic quality checks for training data curation.

Outlook

Outlook Summary

Embodied world models are likely to become closed-loop training engines rather than offline prediction systems. Near-term work will connect imagined futures more directly to reinforcement learning, planning, and action selection. Progress will also depend on physically grounded representations and realistic testbeds that show whether prediction quality improves robot execution, evaluation, and continual learning.

Infographic (English)

Embodied World Models for Robot Learning outlook infographic

Three-Year Movement

The standard path turns embodied world models from video predictors into validation engines for robot learning. The core mechanism is correlation: imagined outcomes must reliably indicate which policy or trajectory will work on a real robot. Better-looking video is not enough if the model gives the wrong ranking to candidate actions.

During the first year, teams will compare simulated rollouts with held-out robot trials. Studies will focus on failures such as collisions, failed grasps, and long-horizon drift. Depth, keypoints, and occupancy maps may help expose errors that visually realistic predictions conceal. A key threshold will be repeated evidence that world-model screening reduces the physical trials needed to reach the same task success rate.

In the second year, successful protocols should spread across a limited range of manipulation and navigation systems. Each test will need a clear operating envelope covering the robot, its sensors, and the task setting. Disagreements between the model and the robot will become valuable training cases, creating a feedback loop that improves both evaluation and later policy updates.

By the third year, shared test suites could influence how bounded robot systems are compared and admitted into operational workflows. Integrated systems may combine simulated rollouts, action proposals, and targeted retraining while keeping physical spot checks authoritative. One monitoring cue is whether hidden robot trials continue to confirm policy rankings after changes in hardware or surroundings. The scenario weakens if rankings fail to transfer, if modest sensor changes erase the gains, or if simpler testing methods reduce physical trials just as effectively. The likely result is simulator-like validation for defined robot and task classes, not universal certification of robot competence.

The contender path uses embodied world models for coverage-guided testing of robot policies. This resembles software fuzzing, where a system is exposed to varied inputs to uncover failures. Here, the model changes scenes or action sequences to find cases such as collisions, dropped objects, or failed recovery. Physical replay remains the authoritative check, so the model only needs to rank and generate useful edge cases rather than simulate everything perfectly.

During the first year, research groups will connect action-conditioned prediction to explicit failure conditions. Geometry checks based on depth or occupancy can remove obviously impossible cases before robot testing. Early evaluation will measure reproducible failures found, confirmation on hardware, and policy improvement from validated cases. The main trigger will be replicated evidence that targeted failure generation produces more useful discoveries than the same amount of broad synthetic experience.

In the second year, persistent regression suites should become more common. Every confirmed failure will be saved and tested again after a policy update, while the model searches for related or harder variants. Teams will also measure behavioral coverage, meaning which task states and recovery paths have actually been exercised. Shared interfaces could support reusable failure libraries across several policies and robot types, although transfer across different machines will remain difficult.

By the third year, this process could become a continuous assurance layer for selected robot operations. World models would propose and prioritize tests, while conventional simulators, limited physical replay, and human review determine whether each result is credible. Regression histories could then support deployment approval and incident analysis without replacing real-world checks. A useful monitoring cue is the physical replay rate: generated failures should be confirmed often enough to justify the testing pipeline. The path weakens if hard cases rarely reproduce, remain specific to one installation, or improve policies less than generic synthetic data. Because robot contact is variable and testing can itself cause harm, the likely outcome is a layered testing system rather than fully automated approval.

The maybe path extends embodied world models into qualified evaluation infrastructure for bounded robot tasks. Its mechanism resembles audited flight simulation: repeatable simulated evidence can reduce some physical trials when real tests are costly or dangerous. This is a harder institutional step than using world models for internal training because independent assessors must trust the measurements.

During the first year, laboratories will test whether imagined outcomes consistently predict real robot results. Seeded rollouts, repeated runs, and real-to-model comparisons will reveal whether results are reproducible. Physical representations based on depth, keypoints, or occupancy can expose geometry errors hidden by realistic video. The important threshold is independent evidence that simulated results predict task performance and safety across more than one setting.

In the second year, progress would depend on whether testing organizations give that evidence practical weight. Researchers may define fidelity classes that distinguish visual realism from physical consistency and closed-loop agreement with robots. Narrow qualification studies could then cover structured navigation or manipulation, with independent services running reproducible tests for developers. A difficult issue will be reassessing small updates without repeating the entire evaluation, since an expensive full review could discourage frequent improvement.

By the third year, a positive branch could produce reference test methods for a few bounded task families and automated requalification pipelines. World-model rollouts might replace selected physical tests, but real robots and runtime safeguards would remain mandatory. Demand for measurable fidelity would encourage better logs of real interactions, which would improve the next generation of evaluation models. A monitoring cue is acceptance of documented cases where synthetic evidence safely reduces a defined part of physical testing. The scenario weakens if results vary across laboratories, fail after modest environmental changes, or miss a serious real-world failure. Robots face changing objects and uncertain contact, so permanent hybrid testing is more plausible than broad qualification based mainly on simulation.

1-Year / 3-Year Research-Application Infographic

Integrated scenario outlook infographic

References

  • WoW: Towards a World omniscient World model Through Embodied Interaction - Authors: Xiaowei Chi, Peidong Jia, Chun-Kai Fan, Xiaozhu Ju, Weishi Mi, Kevin Zhang, Zhiyuan Qin, Wanxin Tian, Kuangzhi Ge, Hao Li, Zezhong Qian, Anthony Chen, Qiang Zhou, Yueru Jia, Jiaming Liu, Yong Dai, Qingpo Wuwu, Chengyu Bai, Yu-Kai Wang, Ying Li, Lizhang Chen, Yong Bao, Zhiyuan Jiang, Jiacheng Zhu, Kai Tang, Ruichuan An, Yulin Luo, Qiuxuan Feng, Siyuan Zhou, Chi-min Chan, Chengkai Hou, Wei Xue, Sirui Han, Yike Guo, Shanghang Zhang, Jian Tang, / <See Details on Fugu-MT> / License: CC-BY-4.0
  • GigaBrain-0: A World Model-Powered Vision-Language-Action Model - Authors: GigaBrain Team, Angen Ye, Boyuan Wang, Chaojun Ni, Guan Huang, Guosheng Zhao, Haoyun Li, Jie Li, Jiagang Zhu, Lv Feng, Peng Li, Qiuping Deng, Runqi Ouyang, Wenkang Qin, Xinze Chen, Xiaofeng Wang, Yang Wang, Yifan Li, Yilong Li, Yiran Ding, Yuan Xu, Yun Ye, Yukun Zhou, Zhehao Dong, Zhenan Wang, Zhichao Liu, Zheng Zhu, / <See Details on Fugu-MT> / License: CC-BY-4.0
  • GigaBrain-0.5M*: a VLA That Learns From World Model-Based Reinforcement Learning - Authors: GigaBrain Team, Boyuan Wang, Chaojun Ni, Guan Huang, Guosheng Zhao, Hao Li, Jie Li, Jindi Lv, Jingyu Liu, Lv Feng, Mingming Yu, Peng Li, Qiuping Deng, Tianze Liu, Xinyu Zhou, Xinze Chen, Xiaofeng Wang, Yang Wang, Yifan Li, Yifei Nie, Yilong Li, Yukun Zhou, Yun Ye, Zhichao Liu, Zheng Zhu, / <See Details on Fugu-MT> / License: CC-BY-4.0
This page was created using generative AI such as GPT-5, Claude Opus 4, Gemini 3, Grok 4.6, Fugu Ultra, GLM 5.3, Gemini 3.1 Flash Image, GPT Image 2, and their higher-end successor versions. No guarantee can be made regarding its contents.