Rank-based No-reference Quality Assessment for Face Swapping

• URL: http://arxiv.org/abs/2406.01884v1
• Date: Tue, 4 Jun 2024 01:36:29 GMT
• Title: Rank-based No-reference Quality Assessment for Face Swapping
• Authors: Xinghui Zhou, Wenbo Zhou, Tianyi Wei, Shen Chen, Taiping Yao, Shouhong Ding, Weiming Zhang, Nenghai Yu,
• Abstract summary: The metric of measuring the quality in most face swapping methods relies on several distances between the manipulated images and the source image. We present a novel no-reference image quality assessment (NR-IQA) method specifically designed for face swapping.
• Score: 88.53827937914038
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Face swapping has become a prominent research area in computer vision and image processing due to rapid technological advancements. The metric of measuring the quality in most face swapping methods relies on several distances between the manipulated images and the source image, or the target image, i.e., there are suitable known reference face images. Therefore, there is still a gap in accurately assessing the quality of face interchange in reference-free scenarios. In this study, we present a novel no-reference image quality assessment (NR-IQA) method specifically designed for face swapping, addressing this issue by constructing a comprehensive large-scale dataset, implementing a method for ranking image quality based on multiple facial attributes, and incorporating a Siamese network based on interpretable qualitative comparisons. Our model demonstrates the state-of-the-art performance in the quality assessment of swapped faces, providing coarse- and fine-grained. Enhanced by this metric, an improved face-swapping model achieved a more advanced level with respect to expressions and poses. Extensive experiments confirm the superiority of our method over existing general no-reference image quality assessment metrics and the latest metric of facial image quality assessment, making it well suited for evaluating face swapping images in real-world scenarios.

Related papers

• Dual-Branch Network for Portrait Image Quality Assessment [76.27716058987251]
  We introduce a dual-branch network for portrait image quality assessment (PIQA) We utilize two backbone networks (textiti.e., Swin Transformer-B) to extract the quality-aware features from the entire portrait image and the facial image cropped from it. We leverage LIQE, an image scene classification and quality assessment model, to capture the quality-aware and scene-specific features as the auxiliary features.
  arXiv  Detail & Related papers  (2024-05-14T12:43:43Z)
• DeepFidelity: Perceptual Forgery Fidelity Assessment for Deepfake Detection [67.3143177137102]
  Deepfake detection refers to detecting artificially generated or edited faces in images or videos. We propose a novel Deepfake detection framework named DeepFidelity to adaptively distinguish real and fake faces.
  arXiv  Detail & Related papers  (2023-12-07T07:19:45Z)
• FaceQAN: Face Image Quality Assessment Through Adversarial Noise Exploration [1.217503190366097]
  We propose a novel approach to face image quality assessment, called FaceQAN, that is based on adversarial examples. As such, the proposed approach is the first to link image quality to adversarial attacks. Experimental results show that FaceQAN achieves competitive results, while exhibiting several desirable characteristics.
  arXiv  Detail & Related papers  (2022-12-05T09:37:32Z)
• AdaFace: Quality Adaptive Margin for Face Recognition [56.99208144386127]
  We introduce another aspect of adaptiveness in the loss function, namely the image quality. We propose a new loss function that emphasizes samples of different difficulties based on their image quality. Our method, AdaFace, improves the face recognition performance over the state-of-the-art (SoTA) on four datasets.
  arXiv  Detail & Related papers  (2022-04-03T01:23:41Z)
• FaceQgen: Semi-Supervised Deep Learning for Face Image Quality Assessment [19.928262020265965]
  FaceQgen is a No-Reference Quality Assessment approach for face images. It generates a scalar quality measure related with the face recognition accuracy. It is trained from scratch using the SCface database.
  arXiv  Detail & Related papers  (2022-01-03T17:22:38Z)
• Pixel-Level Face Image Quality Assessment for Explainable Face Recognition [5.858033242850427]
  We introduce the concept of pixel-level face image quality that determines the utility of pixels in a face image for recognition. Given an arbitrary face recognition network, in this work, we propose a training-free approach to assess the pixel-level qualities of a face image.
  arXiv  Detail & Related papers  (2021-10-21T09:12:17Z)
• Joint Face Image Restoration and Frontalization for Recognition [79.78729632975744]
  In real-world scenarios, many factors may harm face recognition performance, e.g., large pose, bad illumination,low resolution, blur and noise. Previous efforts usually first restore the low-quality faces to high-quality ones and then perform face recognition. We propose an Multi-Degradation Face Restoration model to restore frontalized high-quality faces from the given low-quality ones.
  arXiv  Detail & Related papers  (2021-05-12T03:52:41Z)
• Face Image Quality Assessment: A Literature Survey [16.647739693192236]
  This survey provides an overview of the face image quality assessment literature, which predominantly focuses on visible wavelength face image input. A trend towards deep learning based methods is observed, including notable conceptual differences among the recent approaches.
  arXiv  Detail & Related papers  (2020-09-02T14:26:12Z)
• Inducing Predictive Uncertainty Estimation for Face Recognition [102.58180557181643]
  We propose a method for generating image quality training data automatically from'mated-pairs' of face images. We use the generated data to train a lightweight Predictive Confidence Network, termed as PCNet, for estimating the confidence score of a face image.
  arXiv  Detail & Related papers  (2020-09-01T17:52:00Z)
• SER-FIQ: Unsupervised Estimation of Face Image Quality Based on Stochastic Embedding Robustness [15.431761867166]
  We propose a novel concept to measure face quality based on an arbitrary face recognition model. We compare our proposed solution on two face embeddings against six state-of-the-art approaches from academia and industry.
  arXiv  Detail & Related papers  (2020-03-20T16:50:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.