WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation

• URL: http://arxiv.org/abs/2605.25874v1
• Date: Mon, 25 May 2026 14:01:31 GMT
• Title: WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation
• Abstract summary: WBench is a comprehensive benchmark for evaluation of interactive world models.<n>It covers video quality, setting adherence, interaction adherence, consistency, and physics compliance.<n>It contains 289 test cases and 1,058 interaction turns, covering diverse scenes, styles, subjects, and both first- and third-person perspectives.
• Score: 56.5415838759151
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Interactive world models are advancing rapidly, yet existing benchmarks cover only part of the required competencies, leaving no unified standard for systematic evaluation. To fill this gap, we introduce WBench, a comprehensive multi-turn benchmark for interactive world model evaluation along five dimensions, namely video quality, setting adherence, interaction adherence, consistency, and physics compliance. WBench contains 289 test cases and 1,058 interaction turns, where each case specifies a world setting and a multi-turn interaction sequence, covering diverse scenes, styles, subjects, and both first- and third-person perspectives, together with four interaction types, including navigation, subject action, event editing, and perspective switching. For navigation, WBench unifies text, 6-DoF pose, and discrete-action control, enabling evaluation of models with different native input interfaces. Evaluation uses 22 automatic sub-metrics that combine specialist vision models with large multimodal models, and all metrics are validated against human judgments. Across 20 state-of-the-art models, we find that no single model performs strongly across all dimensions. We provide detailed diagnostic insights into the characteristic strengths, weaknesses, and open challenges of each model. Code and data are available at https://github.com/meituan-longcat/WBench.

Related papers

• PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectives [86.3073067035531]
  Video world models simulate future states conditioned on current observations and user actions.<n>Recent systems have demonstrated impressive video consistency and action controllability over long sequences.<n>But fairly comparing these interactive models remains challenging.<n>We employ multi-modal Agent Players to interact with world models toward specified long-horizon objectives.
  arXiv  Detail & Related papers  (2026-08-13T17:59:30Z)
• iWorld-Bench: A Benchmark for Interactive World Models with a Unified Action Generation Framework [27.208236690012914]
  iWorld-Bench is a benchmark for training and testing world models on interaction-related abilities.<n>We construct a diverse dataset with 330k video clips and select 2.1k high-quality samples covering varied perspectives, weather, and scenes.
  arXiv  Detail & Related papers  (2026-05-05T16:30:03Z)
• WorldMark: A Unified Benchmark Suite for Interactive Video World Models [29.83820642224732]
  We introduce WorldMark, the first benchmark that provides a common playing field for interactive Image-to-Video world models.<n>WorldMark contributes: (1) a unified action-mapping layer that translates a shared WASD-style action vocabulary into each model's native control format, enabling apples-to-apples comparison across six major models on identical scenes and trajectories; (2) a hierarchical test suite of 500 evaluation cases covering first- and third-person viewpoints, photorealistic and stylized scenes, and three difficulty tiers from Easy to Hard spanning 20-60s; and (3) a modular evaluation toolkit for Visual Quality, Control Alignment, and World Consistency
  arXiv  Detail & Related papers  (2026-04-23T13:50:47Z)
• Omni-WorldBench: Towards a Comprehensive Interaction-Centric Evaluation for World Models [39.648000265543445]
  Video-based world models have emerged along two dominant paradigms: video generation and 3D reconstruction.<n>We argue that the future of world modeling lies in 4D generation, which jointly models spatial structure and temporal evolution.<n>We propose Omni--WorldBench, a benchmark specifically designed to evaluate the interactive response capabilities of world models in 4D settings.
  arXiv  Detail & Related papers  (2026-03-23T17:10:29Z)
• The FACTS Leaderboard: A Comprehensive Benchmark for Large Language Model Factuality [70.45240108873001]
  The FACTS Leaderboard is an online leaderboard suite that comprehensively evaluates the ability of language models to generate factually accurate text.<n>The suite provides a holistic measure of factuality by aggregating the performance of models on four distinct sub-leaderboards.
  arXiv  Detail & Related papers  (2025-12-11T16:35:14Z)
• PointArena: Probing Multimodal Grounding Through Language-Guided Pointing [79.80132157576978]
  Pointing serves as a fundamental and intuitive mechanism for grounding language within visual contexts.<n>We introduce PointArena, a comprehensive platform for evaluating multimodal pointing across diverse reasoning scenarios.
  arXiv  Detail & Related papers  (2025-05-15T06:04:42Z)
• MT-Eval: A Multi-Turn Capabilities Evaluation Benchmark for Large Language Models [70.92847554971065]
  We introduce MT-Eval, a comprehensive benchmark designed to evaluate multi-turn conversational abilities. By analyzing human-LLM conversations, we categorize interaction patterns into four types: recollection, expansion, refinement, and follow-up. Our evaluation of 11 well-known LLMs shows that while closed-source models generally surpass open-source ones, certain open-source models exceed GPT-3.5-Turbo in specific tasks.
  arXiv  Detail & Related papers  (2024-01-30T04:50:28Z)
• MultiViz: An Analysis Benchmark for Visualizing and Understanding Multimodal Models [103.9987158554515]
  MultiViz is a method for analyzing the behavior of multimodal models by scaffolding the problem of interpretability into 4 stages. We show that the complementary stages in MultiViz together enable users to simulate model predictions, assign interpretable concepts to features, perform error analysis on model misclassifications, and use insights from error analysis to debug models.
  arXiv  Detail & Related papers  (2022-06-30T18:42:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.