Advancing Zero-Shot Digital Human Quality Assessment through
Text-Prompted Evaluation
- URL: http://arxiv.org/abs/2307.02808v1
- Date: Thu, 6 Jul 2023 06:55:30 GMT
- Title: Advancing Zero-Shot Digital Human Quality Assessment through
Text-Prompted Evaluation
- Authors: Zicheng Zhang, Wei Sun, Yingjie Zhou, Haoning Wu, Chunyi Li, Xiongkuo
Min, Xiaohong Liu, Guangtao Zhai, Weisi Lin
- Abstract summary: SJTU-H3D is a subjective quality assessment database specifically designed for full-body digital humans.
It comprises 40 high-quality reference digital humans and 1,120 labeled distorted counterparts generated with seven types of distortions.
- Score: 60.873105678086404
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Digital humans have witnessed extensive applications in various domains,
necessitating related quality assessment studies. However, there is a lack of
comprehensive digital human quality assessment (DHQA) databases. To address
this gap, we propose SJTU-H3D, a subjective quality assessment database
specifically designed for full-body digital humans. It comprises 40
high-quality reference digital humans and 1,120 labeled distorted counterparts
generated with seven types of distortions. The SJTU-H3D database can serve as a
benchmark for DHQA research, allowing evaluation and refinement of processing
algorithms. Further, we propose a zero-shot DHQA approach that focuses on
no-reference (NR) scenarios to ensure generalization capabilities while
mitigating database bias. Our method leverages semantic and distortion features
extracted from projections, as well as geometry features derived from the mesh
structure of digital humans. Specifically, we employ the Contrastive
Language-Image Pre-training (CLIP) model to measure semantic affinity and
incorporate the Naturalness Image Quality Evaluator (NIQE) model to capture
low-level distortion information. Additionally, we utilize dihedral angles as
geometry descriptors to extract mesh features. By aggregating these measures,
we introduce the Digital Human Quality Index (DHQI), which demonstrates
significant improvements in zero-shot performance. The DHQI can also serve as a
robust baseline for DHQA tasks, facilitating advancements in the field. The
database and the code are available at https://github.com/zzc-1998/SJTU-H3D.
Related papers
- A No-Reference Quality Assessment Method for Digital Human Head [56.17852258306602]
We develop a novel no-reference (NR) method based on Transformer to deal with digital human quality assessment (DHQA)
Specifically, the front 2D projections of the digital humans are rendered as inputs and the vision transformer (ViT) is employed for the feature extraction.
Then we design a multi-task module to jointly classify the distortion types and predict the perceptual quality levels of digital humans.
arXiv Detail & Related papers (2023-10-25T16:01:05Z) - Geometry-Aware Video Quality Assessment for Dynamic Digital Human [56.17852258306602]
We propose a novel no-reference (NR) geometry-aware video quality assessment method for DDH-QA challenge.
The proposed method achieves state-of-the-art performance on the DDH-QA database.
arXiv Detail & Related papers (2023-10-24T16:34:03Z) - AGIQA-3K: An Open Database for AI-Generated Image Quality Assessment [62.8834581626703]
We build the most comprehensive subjective quality database AGIQA-3K so far.
We conduct a benchmark experiment on this database to evaluate the consistency between the current Image Quality Assessment (IQA) model and human perception.
We believe that the fine-grained subjective scores in AGIQA-3K will inspire subsequent AGI quality models to fit human subjective perception mechanisms.
arXiv Detail & Related papers (2023-06-07T18:28:21Z) - DDH-QA: A Dynamic Digital Humans Quality Assessment Database [55.69700918818879]
We construct a large-scale dynamic digital human quality assessment database with diverse motion content as well as multiple distortions.
Ten types of common motion are employed to drive the DDHs and a total of 800 DDHs are generated in the end.
arXiv Detail & Related papers (2022-12-24T13:35:31Z) - Perceptual Quality Assessment for Digital Human Heads [35.801468849447126]
We propose the first large-scale quality assessment database for 3D scanned digital human heads (DHHs)
The constructed database consists of 55 reference DHHs and 1,540 distorted DHHs along with the subjective perceptual ratings.
The experimental results reveal that the proposed method exhibits state-of-the-art performance among the mainstream FR metrics.
arXiv Detail & Related papers (2022-09-20T06:02:57Z) - Blind Quality Assessment of 3D Dense Point Clouds with Structure Guided
Resampling [71.68672977990403]
We propose an objective point cloud quality index with Structure Guided Resampling (SGR) to automatically evaluate the perceptually visual quality of 3D dense point clouds.
The proposed SGR is a general-purpose blind quality assessment method without the assistance of any reference information.
arXiv Detail & Related papers (2022-08-31T02:42:55Z) - No-Reference Quality Assessment for Colored Point Cloud and Mesh Based
on Natural Scene Statistics [36.017914479449864]
We propose an NSS-based no-reference quality assessment metric for colored 3D models.
Our method is mainly validated on the colored point cloud quality assessment database (SJTU-PCQA) and the colored mesh quality assessment database (CMDM)
arXiv Detail & Related papers (2021-07-05T14:03:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.