PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectives

• URL: http://arxiv.org/abs/2608.13552v2
• Date: Fri, 14 Aug 2026 09:46:54 GMT
• Title: PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectives
• Abstract summary: Video world models simulate future states conditioned on current observations and user actions.<n>Recent systems have demonstrated impressive video consistency and action controllability over long sequences.<n>But fairly comparing these interactive models remains challenging.<n>We employ multi-modal Agent Players to interact with world models toward specified long-horizon objectives.
• Score: 86.3073067035531
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Video world models simulate future states conditioned on current observations and user actions. Recent systems have demonstrated impressive video consistency and action controllability over long sequences. However, fairly comparing these interactive models remains challenging. In practice, a human player typically evaluates a world model by pursuing long-horizon objectives through interaction. For example, a user may turn around 360 degrees to see whether the environment remains consistent, or walk into the water and inspect whether realistic water ripples are generated. The action sequence required to achieve the same objective may vary substantially between models, making fixed action-conditioned evaluation unsuitable for cross-model comparison. To address this, we employ multi-modal Agent Players to interact with world models toward specified long-horizon objectives. Building on this paradigm, we introduce PlayWorld, a benchmark providing 171 scenarios, each with a specified objective. To evaluate performance thoroughly, we assess models along four core dimensions: geometry consistency, interaction fidelity, out-of-sight evolution, and insight evolution. In addition, we incorporate basic ability metrics for video quality and controllability. Experiments across nine state-of-the-art world models reveal that current models remain unreliable on long-horizon interactive objectives, particularly in maintaining spatial consistency and persistent state evolution. Code and data are available at https://github.com/kxding/PlayWorld.

Related papers

• From Pixels to States: Rethinking Interactive World Models as Game Engines [32.66997200688335]
  This paper examines interactive game world modeling along four dimensions: player action control, game state dynamics, state-observation persistence, and real-time interactive generation.<n>We present a scalable data engine for Black Myth: Wukong that collects over 90 hours of gameplay with frame-aligned player actions, ground-truth game states, and visual observations.
  arXiv  Detail & Related papers  (2026-07-15T17:48:48Z)
• Trimming the Long-Tail of Visual World Modeling Evaluation [67.17191772079316]
  We introduce Tailor-Bench, a benchmark that challenges world models to simulate irregular physical interactions.<n>Regular scenarios reflect common tool-task pairs, Unconventional scenarios replace conventional tools with attribute-compatible substitutes, and Impossible scenarios introduce attribute-violating tools.<n>Our experimental results reveal a clear long-tail gap in physical world modeling, indicating limited generalization beyond common interactions.
  arXiv  Detail & Related papers  (2026-06-23T07:43:41Z)
• WorldOlympiad: Can Your World Model Survive a Triathlon? [39.69359039523777]
  We introduce WorldOlympiad, a benchmark for diagnosing video-based world models across physical faithfulness, geometric consistency, and interaction fidelity.<n>WorldOlympiad decomposes world-model evaluation into three complementary dimensions.
  arXiv  Detail & Related papers  (2026-06-09T17:24:36Z)
• DisCo: World Models with Discrete Camera Motion Control [79.86256515640231]
  We propose DisCo, a controllable video world model that conditions generation on a compact set of discrete action primitives to improve action separability.<n>DisCo achieves significantly more reliable action following while preserving visual quality.
  arXiv  Detail & Related papers  (2026-06-06T03:50:45Z)
• WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation [56.5415838759151]
  WBench is a comprehensive benchmark for evaluation of interactive world models.<n>It covers video quality, setting adherence, interaction adherence, consistency, and physics compliance.<n>It contains 289 test cases and 1,058 interaction turns, covering diverse scenes, styles, subjects, and both first- and third-person perspectives.
  arXiv  Detail & Related papers  (2026-05-25T14:01:31Z)
• iWorld-Bench: A Benchmark for Interactive World Models with a Unified Action Generation Framework [27.208236690012914]
  iWorld-Bench is a benchmark for training and testing world models on interaction-related abilities.<n>We construct a diverse dataset with 330k video clips and select 2.1k high-quality samples covering varied perspectives, weather, and scenes.
  arXiv  Detail & Related papers  (2026-05-05T16:30:03Z)
• WorldMark: A Unified Benchmark Suite for Interactive Video World Models [29.83820642224732]
  We introduce WorldMark, the first benchmark that provides a common playing field for interactive Image-to-Video world models.<n>WorldMark contributes: (1) a unified action-mapping layer that translates a shared WASD-style action vocabulary into each model's native control format, enabling apples-to-apples comparison across six major models on identical scenes and trajectories; (2) a hierarchical test suite of 500 evaluation cases covering first- and third-person viewpoints, photorealistic and stylized scenes, and three difficulty tiers from Easy to Hard spanning 20-60s; and (3) a modular evaluation toolkit for Visual Quality, Control Alignment, and World Consistency
  arXiv  Detail & Related papers  (2026-04-23T13:50:47Z)
• WorldLens: Full-Spectrum Evaluations of Driving World Models in Real World [100.68103378427567]
  Generative world models are reshaping embodied AI, enabling agents to synthesize realistic 4D driving environments that look convincing but often fail physically or behaviorally.<n>We introduce WorldLens, a full-spectrum benchmark evaluating how well a model builds, understands, and behaves within its generated world.<n>We further construct WorldLens-26K, a large-scale dataset of human-annotated videos with numerical scores and textual rationales, and develop WorldLens-Agent.
  arXiv  Detail & Related papers  (2025-12-11T18:59:58Z)
• GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Control [122.65089441381741]
  We present GEM, a Generalizable Ego-vision Multimodal world model.<n>It predicts future frames using a reference frame, sparse features, human poses, and ego-trajectories.<n>Our dataset is comprised of 4000+ hours of multimodal data across domains like autonomous driving, egocentric human activities, and drone flights.
  arXiv  Detail & Related papers  (2024-12-15T14:21:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.