World Simulation with Video Foundation Models for Physical AI

• URL: http://arxiv.org/abs/2511.00062v1
• Date: Tue, 28 Oct 2025 22:44:13 GMT
• Title: World Simulation with Video Foundation Models for Physical AI
• Abstract summary: We release [Cosmos-Predict2.5] and [Cosmos-Transfer2.5] as versatile tools for scaling embodied intelligence.<n>We release source code, pretrained checkpoints, and curated benchmarks under the NVIDIA Open Model License.
• Score: 181.78265691303315
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce [Cosmos-Predict2.5], the latest generation of the Cosmos World Foundation Models for Physical AI. Built on a flow-based architecture, [Cosmos-Predict2.5] unifies Text2World, Image2World, and Video2World generation in a single model and leverages [Cosmos-Reason1], a Physical AI vision-language model, to provide richer text grounding and finer control of world simulation. Trained on 200M curated video clips and refined with reinforcement learning-based post-training, [Cosmos-Predict2.5] achieves substantial improvements over [Cosmos-Predict1] in video quality and instruction alignment, with models released at 2B and 14B scales. These capabilities enable more reliable synthetic data generation, policy evaluation, and closed-loop simulation for robotics and autonomous systems. We further extend the family with [Cosmos-Transfer2.5], a control-net style framework for Sim2Real and Real2Real world translation. Despite being 3.5$\times$ smaller than [Cosmos-Transfer1], it delivers higher fidelity and robust long-horizon video generation. Together, these advances establish [Cosmos-Predict2.5] and [Cosmos-Transfer2.5] as versatile tools for scaling embodied intelligence. To accelerate research and deployment in Physical AI, we release source code, pretrained checkpoints, and curated benchmarks under the NVIDIA Open Model License at https://github.com/nvidia-cosmos/cosmos-predict2.5 and https://github.com/nvidia-cosmos/cosmos-transfer2.5. We hope these open resources lower the barrier to adoption and foster innovation in building the next generation of embodied intelligence.

Related papers

• Real2Sim2Real for Vision-Language-Action Manipulation: An AMD ROCm-Based Pipeline [13.511902327061856]
  Physical AI is the integration of large vision--action models with embodied agents that act in the real world.<n>This paper presents an end-to-end, fully AMDaccelerated technology stack for embodied manipulation.
  arXiv  Detail & Related papers  (2026-07-25T02:23:34Z)
• Qwen-RobotWorld Technical Report: Unifying Embodied World Modeling through Language-Conditioned Video Generation [88.94027963091206]
  We introduce Qwen-RobotWorld, a language-conditioned video world model for embodied intelligence.<n>It predicts physically grounded future visual trajectories from current observations across robotic manipulation, autonomous driving, indoor navigation, and human-to-robot transfer.
  arXiv  Detail & Related papers  (2026-06-15T17:52:31Z)
• Cosmos 3: Omnimodal World Models for Physical AI [321.3056180868173]
  We introduce Cosmos 3, a family of omnimodal world models designed to jointly process and generate language, image, video, audio, and action sequences.<n>Our evaluation demonstrates that Cosmos 3 unifies a new state-of-the-art across a diverse suite of understanding and generation tasks.
  arXiv  Detail & Related papers  (2026-06-01T19:12:30Z)
• DriveDreamer-Policy: A Geometry-Grounded World-Action Model for Unified Generation and Planning [44.543763428623976]
  We present DriveDreamer-Policy, a unified driving world-action model that integrates depth generation, future video generation, and motion planning.<n>The proposed model produces more coherent imagined futures and more informed driving actions, while maintaining modularity and latency controllable.
  arXiv  Detail & Related papers  (2026-04-02T08:33:18Z)
• Wow, wo, val! A Comprehensive Embodied World Model Evaluation Turing Test [62.17144846428715]
  We introduce the Embodied Turing Test benchmark: WoW-World-Eval (Wow,wo,val)<n>Wow-wo-val examines five core abilities, including perception, planning, prediction, generalization and execution.<n>For the Inverse Dynamic Model Turing Test, we first use an IDM to evaluate the video foundation models' execution accuracy in the real world.
  arXiv  Detail & Related papers  (2026-01-07T17:50:37Z)
• Generative World Modelling for Humanoids: 1X World Model Challenge Technical Report [13.66563834458748]
  1X World Model Challenge introduces an open-source benchmark of real-world humanoid interaction.<n>We adapt the video generation foundation model Wan-2.2 TI2V-5B to video-state-conditioned future frame prediction.<n>Our models achieve 23.0 dB PSNR in the sampling task and a Top-500 CE of 6.6386 in the compression task, securing 1st place in both challenges.
  arXiv  Detail & Related papers  (2025-10-08T14:49:12Z)
• Yume: An Interactive World Generation Model [38.818537395166835]
  Yume aims to use images, text, or videos to create an interactive, realistic, and dynamic world.<n>Method creates a dynamic world from an input image and allows exploration of the world using keyboard actions.
  arXiv  Detail & Related papers  (2025-07-23T17:57:09Z)
• Dreamland: Controllable World Creation with Simulator and Generative Models [32.427050300421115]
  Large-scale video generative models can synthesize diverse and realistic visual content for dynamic world creation.<n>But they often lack element-wise controllability, hindering their use in editing scenes and training embodied AI agents.<n>We propose Dreamland, a hybrid world generation framework combining the granular control of a physics-based simulator and the photorealistic content output of large-scale pretrained generative models.
  arXiv  Detail & Related papers  (2025-06-09T17:59:52Z)
• GR00T N1: An Open Foundation Model for Generalist Humanoid Robots [133.23509142762356]
  General-purpose robots need a versatile body and an intelligent mind.<n>Recent advancements in humanoid robots have shown great promise as a hardware platform for building generalist autonomy.<n>We introduce GR00T N1, an open foundation model for humanoid robots.
  arXiv  Detail & Related papers  (2025-03-18T21:06:21Z)
• Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Control [97.98560001760126]
  We introduce Cosmos-Transfer, a conditional world generation model that can generate world simulations based on multiple spatial control inputs.<n>We conduct evaluations to analyze the proposed model and demonstrate its applications for Physical AI, including robotics2Real and autonomous vehicle data enrichment.
  arXiv  Detail & Related papers  (2025-03-18T17:57:54Z)
• Cosmos World Foundation Model Platform for Physical AI [136.1002343616157]
  We present the Cosmos World Foundation Model Platform to help developers build customized world models for their Physical AI setups.<n>Our platform covers a video curation pipeline, pre-trained world foundation models, examples of post-training of pre-trained world foundation models, and video tokenizers.
  arXiv  Detail & Related papers  (2025-01-07T06:55:50Z)
• VideoPhy: Evaluating Physical Commonsense for Video Generation [93.28748850301949]
  We present VideoPhy, a benchmark designed to assess whether the generated videos follow physical commonsense for real-world activities. We then generate videos conditioned on captions from diverse state-of-the-art text-to-video generative models. Our human evaluation reveals that the existing models severely lack the ability to generate videos adhering to the given text prompts.
  arXiv  Detail & Related papers  (2024-06-05T17:53:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.