Generative Face Parsing Map Guided 3D Face Reconstruction Under Occluded Scenes

• URL: http://arxiv.org/abs/2412.18920v1
• Date: Wed, 25 Dec 2024 14:49:41 GMT
• Title: Generative Face Parsing Map Guided 3D Face Reconstruction Under Occluded Scenes
• Authors: Dapeng Zhao, Yue Qi,
• Abstract summary: A complete face parsing map generation method guided by landmarks is proposed. An excellent anti-occlusion face reconstruction method should ensure the authenticity of the output.
• Score: 4.542616945567623
• License:
• Abstract: Over the past few years, single-view 3D face reconstruction methods can produce beautiful 3D models. Nevertheless,the input of these works is unobstructed faces.We describe a system designed to reconstruct convincing face texture in the case of occlusion.Motivated by parsing facial features,we propose a complete face parsing map generation method guided by landmarks.We estimate the 2D face structure of the reasonable position of the occlusion area,which is used for the construction of 3D texture.An excellent anti-occlusion face reconstruction method should ensure the authenticity of the output,including the topological structure between the eyes,nose, and mouth. We extensively tested our method and its components, qualitatively demonstrating the rationality of our estimated facial structure. We conduct extensive experiments on general 3D face reconstruction tasks as concrete examples to demonstrate the method's superior regulation ability over existing methods often break down.We further provide numerous quantitative examples showing that our method advances both the quality and the robustness of 3D face reconstruction under occlusion scenes.

Related papers

• 3D Face Reconstruction With Geometry Details From a Single Color Image Under Occluded Scenes [4.542616945567623]
  3D face reconstruction technology aims to generate a face stereo model naturally and realistically. Previous deep face reconstruction approaches are typically designed to generate convincing textures. By introducing bump mapping, we successfully added mid-level details to coarse 3D faces.
  arXiv  Detail & Related papers  (2024-12-25T15:16:02Z)
• Generative Landmarks Guided Eyeglasses Removal 3D Face Reconstruction [4.542616945567623]
  Single-view 3D face reconstruction is a fundamental Computer Vision problem of extraordinary difficulty. We present a method for performing a 3D face that removes eyeglasses from a single image.
  arXiv  Detail & Related papers  (2024-12-25T14:57:56Z)
• A Hierarchical Representation Network for Accurate and Detailed Face Reconstruction from In-The-Wild Images [15.40230841242637]
  We present a novel hierarchical representation network (HRN) to achieve accurate and detailed face reconstruction from a single image. Our framework can be extended to a multi-view fashion by considering detail consistency of different views. Our method outperforms the existing methods in both reconstruction accuracy and visual effects.
  arXiv  Detail & Related papers  (2023-02-28T09:24:36Z)
• Perspective Reconstruction of Human Faces by Joint Mesh and Landmark Regression [89.8129467907451]
  We propose to simultaneously reconstruct 3D face mesh in the world space and predict 2D face landmarks on the image plane. Based on the predicted 3D and 2D landmarks, the 6DoF (6 Degrees Freedom) face pose can be easily estimated by the solver.
  arXiv  Detail & Related papers  (2022-08-15T12:32:20Z)
• Facial Geometric Detail Recovery via Implicit Representation [147.07961322377685]
  We present a robust texture-guided geometric detail recovery approach using only a single in-the-wild facial image. Our method combines high-quality texture completion with the powerful expressiveness of implicit surfaces. Our method not only recovers accurate facial details but also decomposes normals, albedos, and shading parts in a self-supervised way.
  arXiv  Detail & Related papers  (2022-03-18T01:42:59Z)
• Segmentation-Reconstruction-Guided Facial Image De-occlusion [48.952656891182826]
  Occlusions are very common in face images in the wild, leading to the degraded performance of face-related tasks. This paper proposes a novel face de-occlusion model based on face segmentation and 3D face reconstruction.
  arXiv  Detail & Related papers  (2021-12-15T10:40:08Z)
• AvatarMe++: Facial Shape and BRDF Inference with Photorealistic Rendering-Aware GANs [119.23922747230193]
  We introduce the first method that is able to reconstruct render-ready 3D facial geometry and BRDF from a single "in-the-wild" image. Our method outperforms the existing arts by a significant margin and reconstructs high-resolution 3D faces from a single low-resolution image.
  arXiv  Detail & Related papers  (2021-12-11T11:36:30Z)
• Inverting Generative Adversarial Renderer for Face Reconstruction [58.45125455811038]
  In this work, we introduce a novel Generative Adversa Renderer (GAR) GAR learns to model the complicated real-world image, instead of relying on the graphics rules, it is capable of producing realistic images. Our method achieves state-of-the-art performances on multiple face reconstruction.
  arXiv  Detail & Related papers  (2021-05-06T04:16:06Z)
• Self-Supervised Monocular 3D Face Reconstruction by Occlusion-Aware Multi-view Geometry Consistency [40.56510679634943]
  We propose a self-supervised training architecture by leveraging the multi-view geometry consistency. We design three novel loss functions for multi-view consistency, including the pixel consistency loss, the depth consistency loss, and the facial landmark-based epipolar loss. Our method is accurate and robust, especially under large variations of expressions, poses, and illumination conditions.
  arXiv  Detail & Related papers  (2020-07-24T12:36:09Z)
• Adaptive 3D Face Reconstruction from a Single Image [45.736818498242016]
  We propose a novel joint 2D and 3D optimization method to adaptively reconstruct 3D face shapes from a single image. Experimental results on multiple datasets demonstrate that our method can generate high-quality reconstruction from a single color image.
  arXiv  Detail & Related papers  (2020-07-08T09:35:26Z)
• AvatarMe: Realistically Renderable 3D Facial Reconstruction "in-the-wild" [105.28776215113352]
  AvatarMe is the first method that is able to reconstruct photorealistic 3D faces from a single "in-the-wild" image with an increasing level of detail. It outperforms the existing arts by a significant margin and reconstructs authentic, 4K by 6K-resolution 3D faces from a single low-resolution image.
  arXiv  Detail & Related papers  (2020-03-30T22:17:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.