TIFace: Improving Facial Reconstruction through Tensorial Radiance Fields and Implicit Surfaces

• URL: http://arxiv.org/abs/2312.09527v1
• Date: Fri, 15 Dec 2023 04:23:20 GMT
• Title: TIFace: Improving Facial Reconstruction through Tensorial Radiance Fields and Implicit Surfaces
• Authors: Ruijie Zhu, Jiahao Chang, Ziyang Song, Jiahuan Yu, Tianzhu Zhang
• Abstract summary: This report describes the solution that secured the first place in the "View Synthesis Challenge for Human Heads" Given the sparse view images of human heads, the objective of this challenge is to synthesize images from novel viewpoints. We propose TI-Face, which improves facial reconstruction through tensorial radiance fields (T-Face) and implicit surfaces (I-Face)
• Score: 34.090466325032686
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This report describes the solution that secured the first place in the "View Synthesis Challenge for Human Heads (VSCHH)" at the ICCV 2023 workshop. Given the sparse view images of human heads, the objective of this challenge is to synthesize images from novel viewpoints. Due to the complexity of textures on the face and the impact of lighting, the baseline method TensoRF yields results with significant artifacts, seriously affecting facial reconstruction. To address this issue, we propose TI-Face, which improves facial reconstruction through tensorial radiance fields (T-Face) and implicit surfaces (I-Face), respectively. Specifically, we employ an SAM-based approach to obtain the foreground mask, thereby filtering out intense lighting in the background. Additionally, we design mask-based constraints and sparsity constraints to eliminate rendering artifacts effectively. The experimental results demonstrate the effectiveness of the proposed improvements and superior performance of our method on face reconstruction. The code will be available at https://github.com/RuijieZhu94/TI-Face.

Related papers

• Face Mask Removal with Region-attentive Face Inpainting [0.7433327915285965]
  We propose a generative face inpainting method to recover/reconstruct the masked part of a face. Our proposed method includes a Multi-scale Channel-Spatial Attention Module (M-CSAM) to mitigate the spatial information loss. We synthesize our own Masked-Faces dataset from the CelebA dataset by incorporating five different types of face masks.
  arXiv  Detail & Related papers  (2024-09-10T20:10:11Z)
• Towards Real-World Blind Face Restoration with Generative Diffusion Prior [69.84480964328465]
  Blind face restoration is an important task in computer vision and has gained significant attention due to its wide-range applications. We propose BFRffusion which is thoughtfully designed to effectively extract features from low-quality face images. We also build a privacy-preserving face dataset called PFHQ with balanced attributes like race, gender, and age.
  arXiv  Detail & Related papers  (2023-12-25T14:16:24Z)
• Learning Representations for Masked Facial Recovery [8.124282476398843]
  pandemic of these recent years has led to a dramatic increase in people wearing protective masks in public venues. One way to address the problem is to revert to face recovery methods as a preprocessing step. We introduce a method that is specific for the recovery of the face image from an image of the same individual wearing a mask.
  arXiv  Detail & Related papers  (2022-12-28T22:22:15Z)
• A Survey of Deep Face Restoration: Denoise, Super-Resolution, Deblur, Artifact Removal [177.21001709272144]
  Face Restoration (FR) aims to restore High-Quality (HQ) faces from Low-Quality (LQ) input images. This paper comprehensively surveys recent advances in deep learning techniques for face restoration.
  arXiv  Detail & Related papers  (2022-11-05T07:08:15Z)
• FaceOcc: A Diverse, High-quality Face Occlusion Dataset for Human Face Extraction [3.8502825594372703]
  Occlusions often occur in face images in the wild, troubling face-related tasks such as landmark detection, 3D reconstruction, and face recognition. This paper proposes a novel face segmentation dataset with manually labeled face occlusions from the CelebA-HQ and the internet. We trained a straightforward face segmentation model but obtained SOTA performance, convincingly demonstrating the effectiveness of the proposed dataset.
  arXiv  Detail & Related papers  (2022-01-20T19:44:18Z)
• AvatarMe++: Facial Shape and BRDF Inference with Photorealistic Rendering-Aware GANs [119.23922747230193]
  We introduce the first method that is able to reconstruct render-ready 3D facial geometry and BRDF from a single "in-the-wild" image. Our method outperforms the existing arts by a significant margin and reconstructs high-resolution 3D faces from a single low-resolution image.
  arXiv  Detail & Related papers  (2021-12-11T11:36:30Z)
• Pro-UIGAN: Progressive Face Hallucination from Occluded Thumbnails [53.080403912727604]
  We propose a multi-stage Progressive Upsampling and Inpainting Generative Adversarial Network, dubbed Pro-UIGAN. It exploits facial geometry priors to replenish and upsample (8*) the occluded and tiny faces. Pro-UIGAN achieves visually pleasing HR faces, reaching superior performance in downstream tasks.
  arXiv  Detail & Related papers  (2021-08-02T02:29:24Z)
• Network Architecture Search for Face Enhancement [82.25775020564654]
  We present a multi-task face restoration network, called Network Architecture Search for Face Enhancement (NASFE) NASFE can enhance poor quality face images containing a single degradation (i.e. noise or blur) or multiple degradations (noise+blur+low-light)
  arXiv  Detail & Related papers  (2021-05-13T19:46:05Z)
• Foreground-guided Facial Inpainting with Fidelity Preservation [7.5089719291325325]
  We propose a foreground-guided facial inpainting framework that can extract and generate facial features using convolutional neural network layers. Specifically, we propose a new loss function with semantic capability reasoning of facial expressions, natural and unnatural features (make-up) Our proposed method achieved comparable quantitative results when compare to the state of the art but qualitatively, it demonstrated high-fidelity preservation of facial components.
  arXiv  Detail & Related papers  (2021-05-07T15:50:58Z)
• HiFaceGAN: Face Renovation via Collaborative Suppression and Replenishment [63.333407973913374]
  "Face Renovation"(FR) is a semantic-guided generation problem. "HiFaceGAN" is a multi-stage framework containing several nested CSR units. experiments on both synthetic and real face images have verified the superior performance of HiFaceGAN.
  arXiv  Detail & Related papers  (2020-05-11T11:33:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.