Generative Landmarks Guided Eyeglasses Removal 3D Face Reconstruction

• URL: http://arxiv.org/abs/2412.19848v1
• Date: Wed, 25 Dec 2024 14:57:56 GMT
• Title: Generative Landmarks Guided Eyeglasses Removal 3D Face Reconstruction
• Authors: Dapeng Zhao, Yue Qi,
• Abstract summary: Single-view 3D face reconstruction is a fundamental Computer Vision problem of extraordinary difficulty. We present a method for performing a 3D face that removes eyeglasses from a single image.
• Score: 4.542616945567623
• License:
• Abstract: Single-view 3D face reconstruction is a fundamental Computer Vision problem of extraordinary difficulty. Current systems often assume the input is unobstructed faces which makes their method not suitable for in-the-wild conditions. We present a method for performing a 3D face that removes eyeglasses from a single image. Existing facial reconstruction methods fail to remove eyeglasses automatically for generating a photo-realistic 3D face "in-the-wild".The innovation of our method lies in a process for identifying the eyeglasses area robustly and remove it intelligently. In this work, we estimate the 2D face structure of the reasonable position of the eyeglasses area, which is used for the construction of 3D texture. An excellent anti-eyeglasses face reconstruction method should ensure the authenticity of the output, including the topological structure between the eyes, nose, and mouth. We achieve this via a deep learning architecture that performs direct regression of a 3DMM representation of the 3D facial geometry from a single 2D image. We also demonstrate how the related face parsing task can be incorporated into the proposed framework and help improve reconstruction quality. We conduct extensive experiments on existing 3D face reconstruction tasks as concrete examples to demonstrate the method's superior regulation ability over existing methods often break down.

Related papers

• 3D Face Reconstruction With Geometry Details From a Single Color Image Under Occluded Scenes [4.542616945567623]
  3D face reconstruction technology aims to generate a face stereo model naturally and realistically. Previous deep face reconstruction approaches are typically designed to generate convincing textures. By introducing bump mapping, we successfully added mid-level details to coarse 3D faces.
  arXiv  Detail & Related papers  (2024-12-25T15:16:02Z)
• Generative Face Parsing Map Guided 3D Face Reconstruction Under Occluded Scenes [4.542616945567623]
  A complete face parsing map generation method guided by landmarks is proposed. An excellent anti-occlusion face reconstruction method should ensure the authenticity of the output.
  arXiv  Detail & Related papers  (2024-12-25T14:49:41Z)
• Pruning-based Topology Refinement of 3D Mesh using a 2D Alpha Mask [6.103988053817792]
  We present a method to refine the topology of any 3D mesh through a face-pruning strategy. Our solution leverages a differentiable that renders each face as a 2D soft map. Because our module is agnostic to the network that produces the 3D mesh, it can be easily plugged into any self-supervised image-based 3D reconstruction pipeline.
  arXiv  Detail & Related papers  (2022-10-17T14:51:38Z)
• Perspective Reconstruction of Human Faces by Joint Mesh and Landmark Regression [89.8129467907451]
  We propose to simultaneously reconstruct 3D face mesh in the world space and predict 2D face landmarks on the image plane. Based on the predicted 3D and 2D landmarks, the 6DoF (6 Degrees Freedom) face pose can be easily estimated by the solver.
  arXiv  Detail & Related papers  (2022-08-15T12:32:20Z)
• AvatarMe++: Facial Shape and BRDF Inference with Photorealistic Rendering-Aware GANs [119.23922747230193]
  We introduce the first method that is able to reconstruct render-ready 3D facial geometry and BRDF from a single "in-the-wild" image. Our method outperforms the existing arts by a significant margin and reconstructs high-resolution 3D faces from a single low-resolution image.
  arXiv  Detail & Related papers  (2021-12-11T11:36:30Z)
• Face Super-Resolution Guided by 3D Facial Priors [92.23902886737832]
  We propose a novel face super-resolution method that explicitly incorporates 3D facial priors which grasp the sharp facial structures. Our work is the first to explore 3D morphable knowledge based on the fusion of parametric descriptions of face attributes. The proposed 3D priors achieve superior face super-resolution results over the state-of-the-arts.
  arXiv  Detail & Related papers  (2020-07-18T15:26:07Z)
• Adaptive 3D Face Reconstruction from a Single Image [45.736818498242016]
  We propose a novel joint 2D and 3D optimization method to adaptively reconstruct 3D face shapes from a single image. Experimental results on multiple datasets demonstrate that our method can generate high-quality reconstruction from a single color image.
  arXiv  Detail & Related papers  (2020-07-08T09:35:26Z)
• Deep 3D Portrait from a Single Image [54.634207317528364]
  We present a learning-based approach for recovering the 3D geometry of human head from a single portrait image. A two-step geometry learning scheme is proposed to learn 3D head reconstruction from in-the-wild face images. We evaluate the accuracy of our method both in 3D and with pose manipulation tasks on 2D images.
  arXiv  Detail & Related papers  (2020-04-24T08:55:37Z)
• AvatarMe: Realistically Renderable 3D Facial Reconstruction "in-the-wild" [105.28776215113352]
  AvatarMe is the first method that is able to reconstruct photorealistic 3D faces from a single "in-the-wild" image with an increasing level of detail. It outperforms the existing arts by a significant margin and reconstructs authentic, 4K by 6K-resolution 3D faces from a single low-resolution image.
  arXiv  Detail & Related papers  (2020-03-30T22:17:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.