WorldScore: A Unified Evaluation Benchmark for World Generation

• URL: http://arxiv.org/abs/2504.00983v1
• Date: Tue, 01 Apr 2025 17:20:23 GMT
• Title: WorldScore: A Unified Evaluation Benchmark for World Generation
• Abstract summary: We introduce the WorldScore benchmark, the first unified benchmark for world generation.<n>WorldScore benchmark encompasses a curated dataset of 3,000 test examples.<n>The WorldScore metrics evaluate generated worlds through three key aspects: controllability, quality, and dynamics.
• Score: 26.259206146192817
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce the WorldScore benchmark, the first unified benchmark for world generation. We decompose world generation into a sequence of next-scene generation tasks with explicit camera trajectory-based layout specifications, enabling unified evaluation of diverse approaches from 3D and 4D scene generation to video generation models. The WorldScore benchmark encompasses a curated dataset of 3,000 test examples that span diverse worlds: static and dynamic, indoor and outdoor, photorealistic and stylized. The WorldScore metrics evaluate generated worlds through three key aspects: controllability, quality, and dynamics. Through extensive evaluation of 19 representative models, including both open-source and closed-source ones, we reveal key insights and challenges for each category of models. Our dataset, evaluation code, and leaderboard can be found at https://haoyi-duan.github.io/WorldScore/

Related papers

• WorldArena 2.0: Extending Embodied World Model Benchmarking on Modality, Functionality and Platform [130.8118909583659]
  We introduce WorldArena 2.0, an expanded benchmark for embodied world model evaluation.<n>WorldArena 2.0 systematically broadens embodied world model evaluation along three dimensions: modality, functionality, and platform.<n>Under a standardized protocol, WorldArena 2.0 comprehensively evaluates perceptual quality, interactive utility, and cross-platform performance.
  arXiv  Detail & Related papers  (2026-05-18T06:18:21Z)
• Is Your Driving World Model an All-Around Player? [107.28582946012818]
  We introduce WorldLens, a unified benchmark that measures world-model fidelity across the full spectrum.<n>Our evaluation of six representative models reveals that no existing approach dominates across all axes.<n>To bridge algorithmic metrics with human perception, we contribute WorldLens-26K, a 26,808-entry human-annotated preference dataset.
  arXiv  Detail & Related papers  (2026-05-11T17:05:49Z)
• iWorld-Bench: A Benchmark for Interactive World Models with a Unified Action Generation Framework [27.208236690012914]
  iWorld-Bench is a benchmark for training and testing world models on interaction-related abilities.<n>We construct a diverse dataset with 330k video clips and select 2.1k high-quality samples covering varied perspectives, weather, and scenes.
  arXiv  Detail & Related papers  (2026-05-05T16:30:03Z)
• WorldMark: A Unified Benchmark Suite for Interactive Video World Models [29.83820642224732]
  We introduce WorldMark, the first benchmark that provides a common playing field for interactive Image-to-Video world models.<n>WorldMark contributes: (1) a unified action-mapping layer that translates a shared WASD-style action vocabulary into each model's native control format, enabling apples-to-apples comparison across six major models on identical scenes and trajectories; (2) a hierarchical test suite of 500 evaluation cases covering first- and third-person viewpoints, photorealistic and stylized scenes, and three difficulty tiers from Easy to Hard spanning 20-60s; and (3) a modular evaluation toolkit for Visual Quality, Control Alignment, and World Consistency
  arXiv  Detail & Related papers  (2026-04-23T13:50:47Z)
• HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds [50.80574129942699]
  HY-World 2.0 is a multi-modal world model framework that advances our prior project HYWorld 1.0.<n>We introduce key innovations to enhance panorama fidelity, enable 3D scene understanding and planning, and upgrade WorldStereo.<n>Also, we introduce WorldLens, a high-performance 3DGS rendering platform featuring a flexible engine-agnostic architecture.
  arXiv  Detail & Related papers  (2026-04-15T17:59:17Z)
• WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Models [114.95269118652163]
  We introduce WorldArena, a unified benchmark designed to evaluate embodied world models across both perceptual and functional dimensions.<n>WorldArena assesses models through three dimensions: video perception quality, measured with 16 metrics across six sub-dimensions; embodied task functionality, which evaluates world models as data engines, policy evaluators, and action planners integrating with subjective human evaluation.<n>Through extensive experiments on 14 representative models, we reveal a significant perception-functionality gap, showing that high visual quality does not necessarily translate into strong embodied task capability.
  arXiv  Detail & Related papers  (2026-02-09T18:09:20Z)
• WorldLens: Full-Spectrum Evaluations of Driving World Models in Real World [100.68103378427567]
  Generative world models are reshaping embodied AI, enabling agents to synthesize realistic 4D driving environments that look convincing but often fail physically or behaviorally.<n>We introduce WorldLens, a full-spectrum benchmark evaluating how well a model builds, understands, and behaves within its generated world.<n>We further construct WorldLens-26K, a large-scale dataset of human-annotated videos with numerical scores and textual rationales, and develop WorldLens-Agent.
  arXiv  Detail & Related papers  (2025-12-11T18:59:58Z)
• WorldPrediction: A Benchmark for High-level World Modeling and Long-horizon Procedural Planning [52.36434784963598]
  We introduce WorldPrediction, a video-based benchmark for evaluating world modeling and procedural planning capabilities of different AI models.<n>We show that current frontier models barely achieve 57% accuracy on WorldPrediction-WM and 38% on WorldPrediction-PP whereas humans are able to solve both tasks perfectly.
  arXiv  Detail & Related papers  (2025-06-04T18:22:40Z)
• E3D-Bench: A Benchmark for End-to-End 3D Geometric Foundation Models [78.1674905950243]
  We present the first comprehensive benchmark for 3D geometric foundation models (GFMs)<n>GFMs directly predict dense 3D representations in a single feed-forward pass, eliminating the need for slow or unavailable precomputed camera parameters.<n>We evaluate 16 state-of-the-art GFMs, revealing their strengths and limitations across tasks and domains.<n>All code, evaluation scripts, and processed data will be publicly released to accelerate research in 3D spatial intelligence.
  arXiv  Detail & Related papers  (2025-06-02T17:53:09Z)
• 3DGen-Bench: Comprehensive Benchmark Suite for 3D Generative Models [94.48803082248872]
  3D generation is experiencing rapid advancements, while the development of 3D evaluation has not kept pace.<n>We develop a large-scale human preference dataset 3DGen-Bench.<n>We then train a CLIP-based scoring model, 3DGen-Score, and a MLLM-based automatic evaluator, 3DGen-Eval.
  arXiv  Detail & Related papers  (2025-03-27T17:53:00Z)
• SimWorld: A Unified Benchmark for Simulator-Conditioned Scene Generation via World Model [1.3700170633913733]
  This paper proposes a simulator-conditioned scene generation engine based on world model.<n>By constructing a simulation system consistent with real-world scenes, simulation data and labels, which serve as the conditions for data generation in the world model, for any scenes can be collected.<n>Results show that these generated images significantly improve downstream perception models performance.
  arXiv  Detail & Related papers  (2025-03-18T06:41:02Z)
• What Are You Doing? A Closer Look at Controllable Human Video Generation [73.89117620413724]
  What Are You Doing?' is a new benchmark for evaluation of controllable image-to-video generation of humans.<n>It consists of 1,544 captioned videos that have been meticulously collected and annotated with 56 fine-grained categories.<n>We perform in-depth analyses of seven state-of-the-art models in controllable image-to-video generation.
  arXiv  Detail & Related papers  (2025-03-06T17:59:29Z)
• WorldModelBench: Judging Video Generation Models As World Models [57.776769550453594]
  Video generation models have rapidly progressed, positioning themselves as video world models capable of supporting decision-making applications like robotics and autonomous driving.<n>Current benchmarks fail to rigorously evaluate these claims, focusing only on general video quality.<n>We propose WorldModelBench, a benchmark designed to evaluate the world modeling capabilities of video generation models in application-driven domains.
  arXiv  Detail & Related papers  (2025-02-28T03:58:23Z)
• Benchmarking and Learning Multi-Dimensional Quality Evaluator for Text-to-3D Generation [26.0726219629689]
  Text-to-3D generation has achieved remarkable progress in recent years, yet evaluating these methods remains challenging. Existing benchmarks lack fine-grained evaluation on different prompt categories and evaluation dimensions. We first propose a comprehensive benchmark named MATE-3D. The benchmark contains eight well-designed prompt categories that cover single and multiple object generation, resulting in 1,280 generated textured meshes.
  arXiv  Detail & Related papers  (2024-12-15T12:41:44Z)
• From an Image to a Scene: Learning to Imagine the World from a Million 360 Videos [71.22810401256234]
  Three-dimensional (3D) understanding of objects and scenes play a key role in humans' ability to interact with the world.<n>Large scale synthetic and object-centric 3D datasets have shown to be effective in training models that have 3D understanding of objects.<n>We introduce 360-1M, a 360 video dataset, and a process for efficiently finding corresponding frames from diverse viewpoints at scale.
  arXiv  Detail & Related papers  (2024-12-10T18:59:44Z)
• VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Models [111.5892290894904]
  VBench is a benchmark suite that dissects "video generation quality" into specific, hierarchical, and disentangled dimensions. We provide a dataset of human preference annotations to validate our benchmarks' alignment with human perception. VBench++ supports evaluating text-to-video and image-to-video.
  arXiv  Detail & Related papers  (2024-11-20T17:54:41Z)
• VBench: Comprehensive Benchmark Suite for Video Generative Models [100.43756570261384]
  VBench is a benchmark suite that dissects "video generation quality" into specific, hierarchical, and disentangled dimensions. We provide a dataset of human preference annotations to validate our benchmarks' alignment with human perception. We will open-source VBench, including all prompts, evaluation methods, generated videos, and human preference annotations.
  arXiv  Detail & Related papers  (2023-11-29T18:39:01Z)
• Stanford-ORB: A Real-World 3D Object Inverse Rendering Benchmark [31.357709524280022]
  Stanford-ORB is a new real-world 3D Object inverse Rendering Benchmark. Recent advances in inverse rendering have enabled a wide range of real-world applications in 3D content generation.
  arXiv  Detail & Related papers  (2023-10-24T17:57:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.