Exploring AIGC Video Quality: A Focus on Visual Harmony, Video-Text Consistency and Domain Distribution Gap

• URL: http://arxiv.org/abs/2404.13573v2
• Date: Sat, 27 Apr 2024 15:10:55 GMT
• Title: Exploring AIGC Video Quality: A Focus on Visual Harmony, Video-Text Consistency and Domain Distribution Gap
• Authors: Bowen Qu, Xiaoyu Liang, Shangkun Sun, Wei Gao,
• Abstract summary: We categorize the assessment of AIGC video quality into three dimensions: visual harmony, video-text consistency, and domain distribution gap. For each dimension, we design specific modules to provide a comprehensive quality assessment of AIGC videos. Our research identifies significant variations in visual quality, fluidity, and style among videos generated by different text-to-video models.
• Score: 4.922783970210658
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The recent advancements in Text-to-Video Artificial Intelligence Generated Content (AIGC) have been remarkable. Compared with traditional videos, the assessment of AIGC videos encounters various challenges: visual inconsistency that defy common sense, discrepancies between content and the textual prompt, and distribution gap between various generative models, etc. Target at these challenges, in this work, we categorize the assessment of AIGC video quality into three dimensions: visual harmony, video-text consistency, and domain distribution gap. For each dimension, we design specific modules to provide a comprehensive quality assessment of AIGC videos. Furthermore, our research identifies significant variations in visual quality, fluidity, and style among videos generated by different text-to-video models. Predicting the source generative model can make the AIGC video features more discriminative, which enhances the quality assessment performance. The proposed method was used in the third-place winner of the NTIRE 2024 Quality Assessment for AI-Generated Content - Track 2 Video, demonstrating its effectiveness. Code will be available at https://github.com/Coobiw/TriVQA.

Related papers

• CLIPVQA:Video Quality Assessment via CLIP [56.94085651315878]
  We propose an efficient CLIP-based Transformer method for the VQA problem ( CLIPVQA) The proposed CLIPVQA achieves new state-of-the-art VQA performance and up to 37% better generalizability than existing benchmark VQA methods.
  arXiv  Detail & Related papers  (2024-07-06T02:32:28Z)
• GAIA: Rethinking Action Quality Assessment for AI-Generated Videos [56.047773400426486]
  Action quality assessment (AQA) algorithms predominantly focus on actions from real specific scenarios and are pre-trained with normative action features. We construct GAIA, a Generic AI-generated Action dataset, by conducting a large-scale subjective evaluation from a novel causal reasoning-based perspective. Based on GAIA, we evaluate a suite of popular text-to-video (T2V) models on their ability to generate visually rational actions. Results show that traditional AQA methods, action-related metrics in recent T2V benchmarks, and mainstream video quality methods correlate poorly with human opinions.
  arXiv  Detail & Related papers  (2024-06-10T08:18:07Z)
• Distinguish Any Fake Videos: Unleashing the Power of Large-scale Data and Motion Features [21.583246378475856]
  We introduce an extensive video dataset designed specifically for AI-Generated Video Detection (GenVidDet) We also present the Dual-Branch 3D Transformer (DuB3D), an innovative and effective method for distinguishing between real and generated videos. DuB3D can distinguish between real and generated video content with 96.77% accuracy, and strong generalization capability even for unseen types.
  arXiv  Detail & Related papers  (2024-05-24T08:26:04Z)
• Subjective-Aligned Dataset and Metric for Text-to-Video Quality Assessment [54.00254267259069]
  We establish the largest-scale Text-to-Video Quality Assessment DataBase (T2VQA-DB) to date. The dataset is composed of 10,000 videos generated by 9 different T2V models. We propose a novel transformer-based model for subjective-aligned Text-to-Video Quality Assessment (T2VQA)
  arXiv  Detail & Related papers  (2024-03-18T16:52:49Z)
• KVQ: Kwai Video Quality Assessment for Short-form Videos [24.5291786508361]
  We establish the first large-scale Kaleidoscope short Video database for Quality assessment, KVQ, which comprises 600 user-uploaded short videos and 3600 processed videos. We propose the first short-form video quality evaluator, i.e., KSVQE, which enables the quality evaluator to identify the quality-determined semantics with the content understanding of large vision language models.
  arXiv  Detail & Related papers  (2024-02-11T14:37:54Z)
• Perceptual Video Quality Assessment: A Survey [63.61214597655413]
  Perceptual video quality assessment plays a vital role in the field of video processing. Various subjective and objective video quality assessment studies have been conducted over the past two decades. This survey provides an up-to-date and comprehensive review of these video quality assessment studies.
  arXiv  Detail & Related papers  (2024-02-05T16:13:52Z)
• AIGCBench: Comprehensive Evaluation of Image-to-Video Content Generated by AI [1.1035305628305816]
  This paper introduces AIGCBench, a pioneering comprehensive benchmark designed to evaluate a variety of video generation tasks. A varied and open-domain image-text dataset that evaluates different state-of-the-art algorithms under equivalent conditions. We employ a novel text combiner and GPT-4 to create rich text prompts, which are then used to generate images via advanced Text-to-Image models.
  arXiv  Detail & Related papers  (2024-01-03T10:08:40Z)
• Towards Explainable In-the-Wild Video Quality Assessment: A Database and a Language-Prompted Approach [52.07084862209754]
  We collect over two million opinions on 4,543 in-the-wild videos on 13 dimensions of quality-related factors. Specifically, we ask the subjects to label among a positive, a negative, and a neutral choice for each dimension. These explanation-level opinions allow us to measure the relationships between specific quality factors and abstract subjective quality ratings.
  arXiv  Detail & Related papers  (2023-05-22T05:20:23Z)
• Towards Robust Text-Prompted Semantic Criterion for In-the-Wild Video Quality Assessment [54.31355080688127]
  We introduce a text-prompted Semantic Affinity Quality Index (SAQI) and its localized version (SAQI-Local) using Contrastive Language-Image Pre-training (CLIP) BVQI-Local demonstrates unprecedented performance, surpassing existing zero-shot indices by at least 24% on all datasets. We conduct comprehensive analyses to investigate different quality concerns of distinct indices, demonstrating the effectiveness and rationality of our design.
  arXiv  Detail & Related papers  (2023-04-28T08:06:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.