LoViF 2026 The First Challenge on Holistic Quality Assessment for 4D World Model (PhyScore)

• URL: http://arxiv.org/abs/2605.05187v1
• Date: Wed, 06 May 2026 17:52:39 GMT
• Title: LoViF 2026 The First Challenge on Holistic Quality Assessment for 4D World Model (PhyScore)
• Abstract summary: The LoViF 2026 PhyScore challenge is a competition on holistic quality assessment of world-model-generated videos.<n>The benchmark dataset contains 1,554 videos generated by seven representative world generative models.<n>This report summarizes the challenge design and provides method-level insights from submitted solutions.
• Score: 130.91833247554004
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper reports on the LoViF 2026 PhyScore challenge, a competition on holistic quality assessment of world-model-generated videos across both 2D and 4D generation settings. The challenge is motivated by a central gap in current evaluation practice: perceptual quality alone is insufficient to judge whether generated dynamics are physically plausible, temporally coherent, and consistent with input conditions. Participants are required to build a metric that jointly predicts four dimensions, i.e., Video Quality, Physical Realism, Condition-Video Alignment, and Temporal Consistency. Depart from that, participants also need to localize physical anomaly timestamps for fine-grained diagnosis. The benchmark dataset contains 1,554 videos generated by seven representative world generative models, organized into three tracks (text-2D, image-to-4D, and video-to-4D) and spanning 26 categories. These categories explicitly cover physics-relevant scenarios, including dynamics, optics, and thermodynamics, together with diverse real-world and creative content. To ensure label reliability, scores and anomaly timestamps are produced through trained human annotation with an additional automated quality-control pass. Evaluation is based on both score prediction and anomaly localization, with a composite protocol that combines TimeStamp_IOU and SRCC/PLCC. This report summarizes the challenge design and provides method-level insights from submitted solutions.

Related papers

• Multi-Dimensional Quality Assessment for AI-Generated Human-Centric Videos: Dataset and Model [95.02470963378896]
  HVEval+ is the largest holistic quality assessment dataset for AI-generated human-centric videos.<n>MoE-Rater is a Mixture-of-Experts (MoE)-inspired and multimodal large language model (MLLM)-based all-in-one method.
  arXiv  Detail & Related papers  (2026-07-18T10:04:18Z)
• WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation [56.5415838759151]
  WBench is a comprehensive benchmark for evaluation of interactive world models.<n>It covers video quality, setting adherence, interaction adherence, consistency, and physics compliance.<n>It contains 289 test cases and 1,058 interaction turns, covering diverse scenes, styles, subjects, and both first- and third-person perspectives.
  arXiv  Detail & Related papers  (2026-05-25T14:01:31Z)
• WorldReasonBench: Human-Aligned Stress Testing of Video Generators as Future World-State Predictors [45.545823511469166]
  We introduce WorldReasonBench, which reframes video generation evaluation as world-state prediction.<n>We evaluate generated videos with a human-aligned two-part methodology.<n>WorldRewardBench is a preference benchmark with approximately 6K expert-annotated pairs over 1.4K videos.
  arXiv  Detail & Related papers  (2026-05-11T12:06:57Z)
• Advancing Reliable Synthetic Video Detection: Insights from the SAFE Challenge [11.327076814364487]
  SAFE: Synthetic Video Detection Challenge was co-located with the textitAuthenticity and Provenance in the Age of Generative AI (APAI) Workshop at ICCV 2025.<n>The competition invited participants to develop and evaluate algorithms capable of distinguishing real from synthetic videos under fully blind evaluation conditions.<n>Our findings highlight measurable progress in cross-generator generalization but also persistent vulnerabilities to post-processing artifacts.
  arXiv  Detail & Related papers  (2026-05-07T20:22:32Z)
• RISE-Video: Can Video Generators Decode Implicit World Rules? [71.92434352963427]
  We present RISE-Video, a pioneering reasoning-oriented benchmark for Text-Image-to-Video (TI2V) synthesis.<n>RISE-Video comprises 467 meticulously human-annotated samples spanning eight rigorous categories.<n>We propose an automated pipeline leveraging Large Multimodal Models (LMMs) to emulate human-centric assessment.
  arXiv  Detail & Related papers  (2026-02-05T18:36:10Z)
• Beyond Rigid: Benchmarking Non-Rigid Video Editing [39.52926053968504]
  We propose NRVBench, the first benchmark designed to evaluate non-rigid video editing.<n>First, we curate a high-quality dataset consisting of 180 non-rigid motion videos from six physics-based categories.<n>Second, we propose NRVE-Acc, a novel evaluation metric based on Vision-Language Models.<n>Third, we introduce a training-free baseline, VM-Edit, which utilizes a dual-region denoising mechanism to achieve structure-aware control.
  arXiv  Detail & Related papers  (2026-01-26T10:28:09Z)
• PhyAVBench: A Challenging Audio Physics-Sensitivity Benchmark for Physically Grounded Text-to-Audio-Video Generation [63.3417467957431]
  Text-to-audio-video (T2AV) generation underpins a wide range of applications demanding realistic audio-visual content.<n>We present PhyAVBench, a challenging audio physics-sensitivity benchmark designed to evaluate the audio physics grounding capabilities of existing T2AV models.<n>Unlike prior benchmarks that primarily focus on audio-video synchronization, PhyAVBench explicitly evaluates models' understanding of the physical mechanisms underlying sound generation.
  arXiv  Detail & Related papers  (2025-12-30T05:22:31Z)
• OmniGround: A Comprehensive Spatio-Temporal Grounding Benchmark for Real-World Complex Scenarios [39.58602686069029]
  We introduce OmniGround, a comprehensive benchmark with 3,475 videos spanning 81 categories and complex real-world queries.<n>We also introduce DeepSTG, a systematic evaluation framework quantifying dataset quality across four complementary dimensions.<n>Experiments demonstrate PG-TAF achieves 25.6% and 35.6% improvements in m_tIoU and m_vIoU with consistent gains across four benchmarks.
  arXiv  Detail & Related papers  (2025-11-21T04:23:04Z)
• VideoVerse: How Far is Your T2V Generator from a World Model? [25.155601280571577]
  VideoVerse is a benchmark that focuses on evaluating whether a T2V model could understand complex temporal causality and world knowledge in the real world.<n>VideoVerse comprises 300 carefully curated prompts, involving 815 events and 793 binary evaluation questions.<n>We perform a systematic evaluation of state-of-the-art open-source and closed-source T2V models on VideoVerse.
  arXiv  Detail & Related papers  (2025-10-09T16:18:20Z)
• VideoScore2: Think before You Score in Generative Video Evaluation [69.43069741467603]
  VideoScore2 is a multi-dimensional, interpretable, and human-aligned framework that explicitly evaluates visual quality, text-to-video alignment, and physical/common-sense consistency.<n>Our model is trained on a large-scale dataset VideoFeedback2 containing 27,168 human-annotated videos.
  arXiv  Detail & Related papers  (2025-09-26T18:09:03Z)
• ChronoMagic-Bench: A Benchmark for Metamorphic Evaluation of Text-to-Time-lapse Video Generation [57.651809298512276]
  ChronoMagic-Bench is a text-to-video (T2V) generation benchmark. It focuses on the model's ability to generate time-lapse videos with significant metamorphic amplitude and temporal coherence. We conduct manual evaluations of ten representative T2V models, revealing their strengths and weaknesses. We create a large-scale ChronoMagic-Pro dataset, containing 460k high-quality pairs of 720p time-lapse videos.
  arXiv  Detail & Related papers  (2024-06-26T17:50:47Z)
• AIM 2020 Challenge on Video Temporal Super-Resolution [118.46127362093135]
  Second AIM challenge on Video Temporal Super-Resolution (VTSR) This paper reports the second AIM challenge on Video Temporal Super-Resolution (VTSR)
  arXiv  Detail & Related papers  (2020-09-28T00:10:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.