VGA-BenchV2: An Expanded Unified Benchmark and Multi-Model Framework for Evaluating Video Aesthetics and Generation Quality

• URL: http://arxiv.org/abs/2608.25452v1
• Date: Wed, 26 Aug 2026 07:16:46 GMT
• Title: VGA-BenchV2: An Expanded Unified Benchmark and Multi-Model Framework for Evaluating Video Aesthetics and Generation Quality
• Abstract summary: VGA-BenchV2 is an extended human-aligned benchmark and optimization framework for jointly evaluating and improving video generation quality and aesthetic value.<n>We curate 1,016 diverse prompts and collect over 60,000 videos generated by 12 mainstream video generation models.<n>We develop a hybrid evaluator architecture consisting of VAQA-Net for continuous aesthetic scoring and two Qwen-based Large Vision-Language Model evaluators, VTag-Net and VGQA-Net, for aesthetic tagging and generation quality assessment.
• Score: 65.0648741395158
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce VGA-BenchV2, an extended human-aligned benchmark and optimization framework for jointly evaluating and improving video generation quality and aesthetic value. Built upon VGA-Bench, VGA-BenchV2 preserves the original fine-grained taxonomy with two primary dimensions-Aesthetic and Generation-and 52 sub-dimensions. Guided by this taxonomy, we curate 1,016 diverse prompts and collect over 60,000 videos generated by 12 mainstream video generation models. More importantly, VGA-BenchV2 substantially expands human-labeled supervision by adding 36,000 task-level annotations, including 16,200 for aesthetic quality, 13,200 for aesthetic tagging, and 6,600 for generation quality, corresponding to 13.46x, 11.15x, and 1.55x scale-ups over VGA-Bench, respectively. Leveraging this enlarged annotation corpus, we develop a hybrid evaluator architecture consisting of VAQA-Net for continuous aesthetic scoring and two Qwen-based Large Vision-Language Model evaluators, VTag-Net and VGQA-Net, for aesthetic tagging and generation quality assessment. Extensive experiments demonstrate strong alignment with human judgments across diverse generation models. Beyond evaluation, VGA-BenchV2 further introduces an evaluation-to-optimization pipeline, where the learned aesthetic evaluator serves as a reward model for reinforcement learning-based generator fine-tuning. This closes the loop from benchmark construction and human supervision to automated evaluation and model optimization, enabling video generators to improve not only in realism but also in aesthetic quality and human preference alignment. Resources are available at https://huggingface.co/datasets/BestiVictoryLab/VGA-Bench.

Related papers

• Multi-Dimensional Quality Assessment for AI-Generated Human-Centric Videos: Dataset and Model [95.02470963378896]
  HVEval+ is the largest holistic quality assessment dataset for AI-generated human-centric videos.<n>MoE-Rater is a Mixture-of-Experts (MoE)-inspired and multimodal large language model (MLLM)-based all-in-one method.
  arXiv  Detail & Related papers  (2026-07-18T10:04:18Z)
• V2V-Bench: A Comprehensive Benchmark for Video-to-Video Generation Evaluation [2.5736307039025057]
  Video-to-video (V2V) generation is difficult to evaluate because outputs must both follow editing instructions and preserve frame-level correspondence with the source video.<n>We introduce V2V-Bench, a benchmark organized into five categories: temporal alignment, structural fidelity, transformation quality, video quality, and semantic alignment.<n>V2V-Bench pairs diverse source videos with challenging editing tasks and evaluates two commercial models, Grok Imagine and Gemini Veo3, and one open-source model, Open Sora 2.
  arXiv  Detail & Related papers  (2026-06-04T03:48:42Z)
• VGA-Bench: A Unified Benchmark and Multi-Model Framework for Video Aesthetics and Generation Quality Evaluation [65.0648741395158]
  VGA-Bench is a unified benchmark for evaluation of video generation quality and aesthetic quality.<n>We design 1,016 diverse prompts and generate a large-scale dataset of over 60,000 videos using 12 video generation models.<n>Our models achieve reliable alignment with human judgments, offering both accuracy and efficiency.
  arXiv  Detail & Related papers  (2026-04-11T09:44:39Z)
• VideoScore2: Think before You Score in Generative Video Evaluation [69.43069741467603]
  VideoScore2 is a multi-dimensional, interpretable, and human-aligned framework that explicitly evaluates visual quality, text-to-video alignment, and physical/common-sense consistency.<n>Our model is trained on a large-scale dataset VideoFeedback2 containing 27,168 human-annotated videos.
  arXiv  Detail & Related papers  (2025-09-26T18:09:03Z)
• VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Models [111.5892290894904]
  VBench is a benchmark suite that dissects "video generation quality" into specific, hierarchical, and disentangled dimensions. We provide a dataset of human preference annotations to validate our benchmarks' alignment with human perception. VBench++ supports evaluating text-to-video and image-to-video.
  arXiv  Detail & Related papers  (2024-11-20T17:54:41Z)
• EvalCrafter: Benchmarking and Evaluating Large Video Generation Models [70.19437817951673]
  We argue that it is hard to judge the large conditional generative models from the simple metrics since these models are often trained on very large datasets with multi-aspect abilities. Our approach involves generating a diverse and comprehensive list of 700 prompts for text-to-video generation. Then, we evaluate the state-of-the-art video generative models on our carefully designed benchmark, in terms of visual qualities, content qualities, motion qualities, and text-video alignment with 17 well-selected objective metrics.
  arXiv  Detail & Related papers  (2023-10-17T17:50:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.