Reconstruct Face from Features Using GAN Generator as a Distribution Constraint

• URL: http://arxiv.org/abs/2206.04295v1
• Date: Thu, 9 Jun 2022 06:11:59 GMT
• Title: Reconstruct Face from Features Using GAN Generator as a Distribution Constraint
• Authors: Xingbo Dong, Zhihui Miao, Lan Ma, Jiajun Shen, Zhe Jin, Zhenhua Guo, Andrew Beng Jin Teoh
• Abstract summary: Face recognition based on the deep convolutional neural networks (CNN) shows superior accuracy performance attributed to the high discriminative features extracted. Yet, the security and privacy of the extracted features from deep learning models (deep features) have been often overlooked. This paper proposes the reconstruction of face images from deep features without accessing the CNN network configurations.
• Score: 17.486032607577577
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Face recognition based on the deep convolutional neural networks (CNN) shows superior accuracy performance attributed to the high discriminative features extracted. Yet, the security and privacy of the extracted features from deep learning models (deep features) have been often overlooked. This paper proposes the reconstruction of face images from deep features without accessing the CNN network configurations as a constrained optimization problem. Such optimization minimizes the distance between the features extracted from the original face image and the reconstructed face image. Instead of directly solving the optimization problem in the image space, we innovatively reformulate the problem by looking for a latent vector of a GAN generator, then use it to generate the face image. The GAN generator serves as a dual role in this novel framework, i.e., face distribution constraint of the optimization goal and a face generator. On top of the novel optimization task, we also propose an attack pipeline to impersonate the target user based on the generated face image. Our results show that the generated face images can achieve a state-of-the-art successful attack rate of 98.0\% on LFW under type-I attack @ FAR of 0.1\%. Our work sheds light on the biometric deployment to meet the privacy-preserving and security policies.

Related papers

• OSDFace: One-Step Diffusion Model for Face Restoration [72.5045389847792]
  Diffusion models have demonstrated impressive performance in face restoration. We propose OSDFace, a novel one-step diffusion model for face restoration. Results demonstrate that OSDFace surpasses current state-of-the-art (SOTA) methods in both visual quality and quantitative metrics.
  arXiv  Detail & Related papers  (2024-11-26T07:07:48Z)
• G2Face: High-Fidelity Reversible Face Anonymization via Generative and Geometric Priors [71.69161292330504]
  Reversible face anonymization seeks to replace sensitive identity information in facial images with synthesized alternatives. This paper introduces Gtextsuperscript2Face, which leverages both generative and geometric priors to enhance identity manipulation. Our method outperforms existing state-of-the-art techniques in face anonymization and recovery, while preserving high data utility.
  arXiv  Detail & Related papers  (2024-08-18T12:36:47Z)
• E2F-Net: Eyes-to-Face Inpainting via StyleGAN Latent Space [4.110419543591102]
  We propose a Generative Adversarial Network (GAN)-based model called Eyes-to-Face Network (E2F-Net) The proposed approach extracts identity and non-identity features from the periocular region using two dedicated encoders. We show that our method successfully reconstructs the whole face with high quality, surpassing current techniques.
  arXiv  Detail & Related papers  (2024-03-18T19:11:34Z)
• Attribute-preserving Face Dataset Anonymization via Latent Code Optimization [64.4569739006591]
  We present a task-agnostic anonymization procedure that directly optimize the images' latent representation in the latent space of a pre-trained GAN. We demonstrate through a series of experiments that our method is capable of anonymizing the identity of the images whilst -- crucially -- better-preserving the facial attributes.
  arXiv  Detail & Related papers  (2023-03-20T17:34:05Z)
• Enhancing Quality of Pose-varied Face Restoration with Local Weak Feature Sensing and GAN Prior [29.17397958948725]
  We propose a well-designed blind face restoration network with generative facial prior. Our model performs superior to the prior art for face restoration and face super-resolution tasks.
  arXiv  Detail & Related papers  (2022-05-28T09:23:48Z)
• SelFSR: Self-Conditioned Face Super-Resolution in the Wild via Flow Field Degradation Network [12.976199676093442]
  We propose a novel domain-adaptive degradation network for face super-resolution in the wild. Our model achieves state-of-the-art performance on both CelebA and real-world face dataset.
  arXiv  Detail & Related papers  (2021-12-20T17:04:00Z)
• Inverting Generative Adversarial Renderer for Face Reconstruction [58.45125455811038]
  In this work, we introduce a novel Generative Adversa Renderer (GAR) GAR learns to model the complicated real-world image, instead of relying on the graphics rules, it is capable of producing realistic images. Our method achieves state-of-the-art performances on multiple face reconstruction.
  arXiv  Detail & Related papers  (2021-05-06T04:16:06Z)
• Riggable 3D Face Reconstruction via In-Network Optimization [58.016067611038046]
  This paper presents a method for riggable 3D face reconstruction from monocular images. It jointly estimates a personalized face rig and per-image parameters including expressions, poses, and illuminations. Experiments demonstrate that our method achieves SOTA reconstruction accuracy, reasonable robustness and generalization ability.
  arXiv  Detail & Related papers  (2021-04-08T03:53:20Z)
• High Resolution Face Editing with Masked GAN Latent Code Optimization [0.0]
  Face editing is a popular research topic in the computer vision community. Recent proposed methods are based on either training a conditional encoder-decoder Generative Adversarial Network (GAN) in an end-to-end fashion or on defining an operation in the latent space of a pre-trained vanilla GAN generator model. We propose a GAN embedding optimization procedure with spatial and semantic constraints.
  arXiv  Detail & Related papers  (2021-03-20T08:39:41Z)
• Black-Box Face Recovery from Identity Features [61.950765357647605]
  We attack the state-of-the-art face recognition system (ArcFace) to test our algorithm. Our algorithm requires a significantly less number of queries compared to the state-of-the-art solution.
  arXiv  Detail & Related papers  (2020-07-27T15:25:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.