Deep Learning-Based Quasi-Conformal Surface Registration for Partial 3D Faces Applied to Facial Recognition

• URL: http://arxiv.org/abs/2405.09880v1
• Date: Thu, 16 May 2024 08:03:41 GMT
• Title: Deep Learning-Based Quasi-Conformal Surface Registration for Partial 3D Faces Applied to Facial Recognition
• Authors: Yuchen Guo, Hanqun Cao, Lok Ming Lui,
• Abstract summary: 3D face registration is an important process in which a 3D face model is aligned and mapped to a template face. This paper presents a novel deep learning-based approach that combines quasi-conformal geometry with deep neural networks for partial face registration.
• Score: 13.426313868579827
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: 3D face registration is an important process in which a 3D face model is aligned and mapped to a template face. However, the task of 3D face registration becomes particularly challenging when dealing with partial face data, where only limited facial information is available. To address this challenge, this paper presents a novel deep learning-based approach that combines quasi-conformal geometry with deep neural networks for partial face registration. The proposed framework begins with a Landmark Detection Network that utilizes curvature information to detect the presence of facial features and estimate their corresponding coordinates. These facial landmark features serve as essential guidance for the registration process. To establish a dense correspondence between the partial face and the template surface, a registration network based on quasiconformal theories is employed. The registration network establishes a bijective quasiconformal surface mapping aligning corresponding partial faces based on detected landmarks and curvature values. It consists of the Coefficients Prediction Network, which outputs the optimal Beltrami coefficient representing the surface mapping. The Beltrami coefficient quantifies the local geometric distortion of the mapping. By controlling the magnitude of the Beltrami coefficient through a suitable activation function, the bijectivity and geometric distortion of the mapping can be controlled. The Beltrami coefficient is then fed into the Beltrami solver network to reconstruct the corresponding mapping. The surface registration enables the acquisition of corresponding regions and the establishment of point-wise correspondence between different partial faces, facilitating precise shape comparison through the evaluation of point-wise geometric differences at these corresponding regions. Experimental results demonstrate the effectiveness of the proposed method.

Related papers

• InfoNorm: Mutual Information Shaping of Normals for Sparse-View Reconstruction [15.900375207144759]
  3D surface reconstruction from multi-view images is essential for scene understanding and interaction. Recent implicit surface representations, such as Neural Radiance Fields (NeRFs) and signed distance functions (SDFs) employ various geometric priors to resolve the lack of observed information. We propose regularizing the geometric modeling by explicitly encouraging the mutual information among surface normals of highly correlated scene points.
  arXiv  Detail & Related papers  (2024-07-17T15:46:25Z)
• Neural Semantic Surface Maps [52.61017226479506]
  We present an automated technique for computing a map between two genus-zero shapes, which matches semantically corresponding regions to one another. Our approach can generate semantic surface-to-surface maps, eliminating manual annotations or any 3D training data requirement.
  arXiv  Detail & Related papers  (2023-09-09T16:21:56Z)
• Semantic-aware One-shot Face Re-enactment with Dense Correspondence Estimation [100.60938767993088]
  One-shot face re-enactment is a challenging task due to the identity mismatch between source and driving faces. This paper proposes to use 3D Morphable Model (3DMM) for explicit facial semantic decomposition and identity disentanglement.
  arXiv  Detail & Related papers  (2022-11-23T03:02:34Z)
• Automatic Landmark Detection and Registration of Brain Cortical Surfaces via Quasi-Conformal Geometry and Convolutional Neural Networks [17.78250777571423]
  We propose a novel framework for the automatic landmark detection and registration of brain cortical surfaces. We first develop a landmark detection network (LD-Net) that allows for the automatic extraction of landmark curves. We then utilize the detected landmarks and quasi-conformal theory for achieving the surface registration.
  arXiv  Detail & Related papers  (2022-08-15T05:47:51Z)
• Topologically Consistent Multi-View Face Inference Using Volumetric Sampling [25.001398662643986]
  ToFu is a geometry inference framework that can produce topologically consistent meshes across identities and expressions. A novel progressive mesh generation network embeds the topological structure of the face in a feature volume. These high-quality assets are readily usable by production studios for avatar creation, animation and physically-based skin rendering.
  arXiv  Detail & Related papers  (2021-10-06T17:55:08Z)
• Deep Implicit Surface Point Prediction Networks [49.286550880464866]
  Deep neural representations of 3D shapes as implicit functions have been shown to produce high fidelity models. This paper presents a novel approach that models such surfaces using a new class of implicit representations called the closest surface-point (CSP) representation.
  arXiv  Detail & Related papers  (2021-06-10T14:31:54Z)
• Progressive Spatio-Temporal Bilinear Network with Monte Carlo Dropout for Landmark-based Facial Expression Recognition with Uncertainty Estimation [93.73198973454944]
  The performance of our method is evaluated on three widely used datasets. It is comparable to that of video-based state-of-the-art methods while it has much less complexity.
  arXiv  Detail & Related papers  (2021-06-08T13:40:30Z)
• Locally Aware Piecewise Transformation Fields for 3D Human Mesh Registration [67.69257782645789]
  We propose piecewise transformation fields that learn 3D translation vectors to map any query point in posed space to its correspond position in rest-pose space. We show that fitting parametric models with poses by our network results in much better registration quality, especially for extreme poses.
  arXiv  Detail & Related papers  (2021-04-16T15:16:09Z)
• DualConv: Dual Mesh Convolutional Networks for Shape Correspondence [44.94765770516059]
  Convolutional neural networks have been extremely successful for 2D images and are readily extended to handle 3D voxel data. In this paper we explore how these networks can be extended to the dual face-based representation of triangular meshes. Our experiments demonstrate that building additionally convolutional models that explicitly leverage the neighborhood size regularity of dual meshes enables learning shape representations that perform on par or better than previous approaches.
  arXiv  Detail & Related papers  (2021-03-23T11:22:47Z)
• LARNet: Lie Algebra Residual Network for Profile Face Recognition [5.968418413932049]
  We propose a novel method with Lie algebra theory to explore how face rotation in the 3D space affects the deep feature generation process of convolutional neural networks (CNNs) We prove that face rotation in the image space is equivalent to an additive residual component in the feature space of CNNs, which is determined solely by the rotation. Our LARNet design consists of a residual for decoding rotation information from input face images, and a gating magnitude to learn rotation for controlling the number of residual components contributing to the feature learning process.
  arXiv  Detail & Related papers  (2021-03-15T05:44:54Z)
• Shape My Face: Registering 3D Face Scans by Surface-to-Surface Translation [75.59415852802958]
  Shape-My-Face (SMF) is a powerful encoder-decoder architecture based on an improved point cloud encoder, a novel visual attention mechanism, graph convolutional decoders with skip connections, and a specialized mouth model. Our model provides topologically-sound meshes with minimal supervision, offers faster training time, has orders of magnitude fewer trainable parameters, is more robust to noise, and can generalize to previously unseen datasets.
  arXiv  Detail & Related papers  (2020-12-16T20:02:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.