Generative World Modelling for Humanoids: 1X World Model Challenge Technical Report

• URL: http://arxiv.org/abs/2510.07092v1
• Date: Wed, 08 Oct 2025 14:49:12 GMT
• Title: Generative World Modelling for Humanoids: 1X World Model Challenge Technical Report
• Abstract summary: 1X World Model Challenge introduces an open-source benchmark of real-world humanoid interaction.<n>We adapt the video generation foundation model Wan-2.2 TI2V-5B to video-state-conditioned future frame prediction.<n>Our models achieve 23.0 dB PSNR in the sampling task and a Top-500 CE of 6.6386 in the compression task, securing 1st place in both challenges.
• Score: 13.66563834458748
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: World models are a powerful paradigm in AI and robotics, enabling agents to reason about the future by predicting visual observations or compact latent states. The 1X World Model Challenge introduces an open-source benchmark of real-world humanoid interaction, with two complementary tracks: sampling, focused on forecasting future image frames, and compression, focused on predicting future discrete latent codes. For the sampling track, we adapt the video generation foundation model Wan-2.2 TI2V-5B to video-state-conditioned future frame prediction. We condition the video generation on robot states using AdaLN-Zero, and further post-train the model using LoRA. For the compression track, we train a Spatio-Temporal Transformer model from scratch. Our models achieve 23.0 dB PSNR in the sampling task and a Top-500 CE of 6.6386 in the compression task, securing 1st place in both challenges.

Related papers

• Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory [53.39687409541093]
  Matrix-Game 3.0 is a memory-augmented interactive world model designed for 720p real-time longform video generation.<n>We introduce systematic improvements across data, model, and inference.<n> Experimental results show that Matrix-Game 3.0 achieves up to 40 FPS real-time generation at 720p resolution with a 5B model.
  arXiv  Detail & Related papers  (2026-04-10T06:00:09Z)
• Disentangled Robot Learning via Separate Forward and Inverse Dynamics Pretraining [28.30092786035367]
  DeFI is a novel framework that Decouples visual Forward and Inverse dynamics pretraining to exploit respective data sources.<n>We introduce the General Forward Dynamics Model (GFDM), pretrained on diverse human and robot videos for future prediction, and the General Inverse Dynamics Model (GIDM), trained via self-supervised learning to infer latent actions from unlabeled video transitions.<n>Experiments on CALVIN ABC-D and SimplerEnv demonstrate state-of-the-art performance, with DeFI achieving an average task length of 4.51 for CALVIN, 51.2% success rate on SimplerEnv-Frac
  arXiv  Detail & Related papers  (2026-03-27T17:20:10Z)
• GigaWorld-Policy: An Efficient Action-Centered World--Action Model [50.107640832046464]
  GigaWorld-Policy is an action-centered WAM that learns 2D pixel-action dynamics while enabling efficient action decoding, with optional video generation.<n> Experimental results on real-world robotic platforms show that GigaWorld-Policy runs 9x faster than the leading WAM baseline, Motus.
  arXiv  Detail & Related papers  (2026-03-18T00:52:02Z)
• DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Control [16.562259973551786]
  We introduce DiT4DiT, an end-to-end Video-Action Model that couples a video Diffusion Transformer with an action Diffusion Transformer.<n>Instead of relying on reconstructed future frames, DiT4DiT extracts intermediate denoising features from the video generation process.<n>It achieves state-of-the-art results, reaching average success rates of 98.6% on LIBERO and 50.8% on RoboCasa GR1.
  arXiv  Detail & Related papers  (2026-03-11T06:03:53Z)
• Causal World Modeling for Robot Control [56.31803788587547]
  Video world models provide the ability to imagine the near future by understanding the causality between actions and visual dynamics.<n>We introduce LingBot-VA, an autoregressive diffusion framework that learns frame prediction and policy execution simultaneously.<n>We evaluate our model on both simulation benchmarks and real-world scenarios, where it shows significant promise in long-horizon manipulation, data efficiency in post-training, and strong generalizability to novel configurations.
  arXiv  Detail & Related papers  (2026-01-29T17:07:43Z)
• Wow, wo, val! A Comprehensive Embodied World Model Evaluation Turing Test [62.17144846428715]
  We introduce the Embodied Turing Test benchmark: WoW-World-Eval (Wow,wo,val)<n>Wow-wo-val examines five core abilities, including perception, planning, prediction, generalization and execution.<n>For the Inverse Dynamic Model Turing Test, we first use an IDM to evaluate the video foundation models' execution accuracy in the real world.
  arXiv  Detail & Related papers  (2026-01-07T17:50:37Z)
• Humanoid World Models: Open World Foundation Models for Humanoid Robotics [2.7781524868854715]
  We introduce Humanoid World Models (HWM), a family of lightweight, open-source models that forecast future egocentric video conditioned on humanoid control tokens.<n>We train two types of generative models, Masked Transformers and Flow-Matching, on 100 hours of humanoid demonstrations.<n>Our parameter-sharing techniques reduce model size by 33-53% with minimal impact on performance or visual fidelity.
  arXiv  Detail & Related papers  (2025-06-01T21:33:36Z)
• WorldModelBench: Judging Video Generation Models As World Models [57.776769550453594]
  Video generation models have rapidly progressed, positioning themselves as video world models capable of supporting decision-making applications like robotics and autonomous driving.<n>Current benchmarks fail to rigorously evaluate these claims, focusing only on general video quality.<n>We propose WorldModelBench, a benchmark designed to evaluate the world modeling capabilities of video generation models in application-driven domains.
  arXiv  Detail & Related papers  (2025-02-28T03:58:23Z)
• D$^2$-World: An Efficient World Model through Decoupled Dynamic Flow [47.361822281431586]
  This report summarizes the second-place solution for the Predictive World Model Challenge held at the CVPR-2024 Workshop on Foundation Models for Autonomous Systems. We introduce D$2$-World, a novel World model that effectively forecasts future point clouds through Decoupled Dynamic flow. Our approach achieves state-of-the-art performance on the OpenScene Predictive World Model benchmark, securing second place, and trains more than 300% faster than the baseline model.
  arXiv  Detail & Related papers  (2024-11-26T01:42:49Z)
• VidMan: Exploiting Implicit Dynamics from Video Diffusion Model for Effective Robot Manipulation [79.00294932026266]
  VidMan is a novel framework that employs a two-stage training mechanism to enhance stability and improve data utilization efficiency. Our framework outperforms state-of-the-art baseline model GR-1 on the CALVIN benchmark, achieving a 11.7% relative improvement, and demonstrates over 9% precision gains on the OXE small-scale dataset.
  arXiv  Detail & Related papers  (2024-11-14T03:13:26Z)
• Copilot4D: Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusion [36.321494200830244]
  Copilot4D is a novel world modeling approach that first tokenizes sensor observations with VQVAE, then predicts the future via discrete diffusion. Our results demonstrate that discrete diffusion on tokenized agent experience can unlock the power of GPT-like unsupervised learning for robotics.
  arXiv  Detail & Related papers  (2023-11-02T06:21:56Z)
• Seer: Language Instructed Video Prediction with Latent Diffusion Models [43.708550061909754]
  Text-conditioned video prediction (TVP) is an essential task to facilitate general robot policy learning. We propose a sample and computation-efficient model, named textbfSeer, by inflating the pretrained text-to-image (T2I) stable diffusion models along the temporal axis. With the adaptable-designed architecture, Seer makes it possible to generate high-fidelity, coherent, and instruction-aligned video frames.
  arXiv  Detail & Related papers  (2023-03-27T03:12:24Z)
• T3VIP: Transformation-based 3D Video Prediction [49.178585201673364]
  We propose a 3D video prediction (T3VIP) approach that explicitly models the 3D motion by decomposing a scene into its object parts. Our model is fully unsupervised, captures the nature of the real world, and the observational cues in image and point cloud domains constitute its learning signals. To the best of our knowledge, our model is the first generative model that provides an RGB-D video prediction of the future for a static camera.
  arXiv  Detail & Related papers  (2022-09-19T15:01:09Z)
• HARP: Autoregressive Latent Video Prediction with High-Fidelity Image Generator [90.74663948713615]
  We train an autoregressive latent video prediction model capable of predicting high-fidelity future frames. We produce high-resolution (256x256) videos with minimal modification to existing models.
  arXiv  Detail & Related papers  (2022-09-15T08:41:57Z)
• Future Frame Prediction for Robot-assisted Surgery [57.18185972461453]
  We propose a ternary prior guided variational autoencoder (TPG-VAE) model for future frame prediction in robotic surgical video sequences. Besides content distribution, our model learns motion distribution, which is novel to handle the small movements of surgical tools.
  arXiv  Detail & Related papers  (2021-03-18T15:12:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.