Towards Geometric-Photometric Joint Alignment for Facial Mesh Registration

• URL: http://arxiv.org/abs/2403.02629v1
• Date: Tue, 5 Mar 2024 03:39:23 GMT
• Title: Towards Geometric-Photometric Joint Alignment for Facial Mesh Registration
• Authors: Xizhi Wang and Yaxiong Wang and Mengjian Li
• Abstract summary: This paper presents a Geometric-Photometric Joint Alignment method, for accurately aligning human expressions by combining geometry and photometric information. Experimental results demonstrate faithful alignment under various expressions, surpassing the conventional ICP-based methods and the state-of-the-art deep learning based method. In practical, our method enhances the efficiency of obtaining topology-consistent face models from multi-view stereo facial scanning.
• Score: 3.588864037082647
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This paper presents a Geometric-Photometric Joint Alignment(GPJA) method, for accurately aligning human expressions by combining geometry and photometric information. Common practices for registering human heads typically involve aligning landmarks with facial template meshes using geometry processing approaches, but often overlook photometric consistency. GPJA overcomes this limitation by leveraging differentiable rendering to align vertices with target expressions, achieving joint alignment in geometry and photometric appearances automatically, without the need for semantic annotation or aligned meshes for training. It features a holistic rendering alignment strategy and a multiscale regularized optimization for robust and fast convergence. The method utilizes derivatives at vertex positions for supervision and employs a gradient-based algorithm which guarantees smoothness and avoids topological defects during the geometry evolution. Experimental results demonstrate faithful alignment under various expressions, surpassing the conventional ICP-based methods and the state-of-the-art deep learning based method. In practical, our method enhances the efficiency of obtaining topology-consistent face models from multi-view stereo facial scanning.

Related papers

• Geometry-aware Feature Matching for Large-Scale Structure from Motion [10.645087195983201]
  We introduce geometry cues in addition to color cues to fill gaps when there is less overlap in large-scale scenarios. Our approach ensures that the denser correspondences from detector-free methods are geometrically consistent and more accurate. It outperforms state-of-the-art feature matching methods on benchmark datasets.
  arXiv  Detail & Related papers  (2024-09-03T21:41:35Z)
• Geometric Artifact Correction for Symmetric Multi-Linear Trajectory CT: Theory, Method, and Generalization [3.343129011758024]
  The existing calibration method for CT is both crude and inefficient. In this paper, we comprehensively and efficiently address the challenging geometric artifacts in CT. We also demonstrate significant generalization to common rotated CT and a variant of CT.
  arXiv  Detail & Related papers  (2024-08-27T13:56:48Z)
• Boundary Constraint-free Biomechanical Model-Based Surface Matching for Intraoperative Liver Deformation Correction [0.6249768559720122]
  In image-guided liver surgery, 3D-3D non-rigid registration methods play a crucial role in estimating the mapping between the preoperative model and the intraoperative surface represented as point clouds. This paper introduces a novel 3D-3D non-rigid registration method. In contrast to the preceding techniques, our method uniquely incorporates the FEM within the surface matching term itself.
  arXiv  Detail & Related papers  (2024-03-15T02:05:20Z)
• Adaptive Surface Normal Constraint for Geometric Estimation from Monocular Images [56.86175251327466]
  We introduce a novel approach to learn geometries such as depth and surface normal from images while incorporating geometric context. Our approach extracts geometric context that encodes the geometric variations present in the input image and correlates depth estimation with geometric constraints. Our method unifies depth and surface normal estimations within a cohesive framework, which enables the generation of high-quality 3D geometry from images.
  arXiv  Detail & Related papers  (2024-02-08T17:57:59Z)
• Efficient Multi-View Inverse Rendering Using a Hybrid Differentiable Rendering Method [19.330797817738542]
  We introduce a novel hybrid differentiable rendering method to efficiently reconstruct the 3D geometry and reflectance of a scene. Our method can produce reconstructions with similar or higher quality than state-of-the-art methods while being more efficient.
  arXiv  Detail & Related papers  (2023-08-19T12:48:10Z)
• Parallax-Tolerant Unsupervised Deep Image Stitching [57.76737888499145]
  We propose UDIS++, a parallax-tolerant unsupervised deep image stitching technique. First, we propose a robust and flexible warp to model the image registration from global homography to local thin-plate spline motion. To further eliminate the parallax artifacts, we propose to composite the stitched image seamlessly by unsupervised learning for seam-driven composition masks.
  arXiv  Detail & Related papers  (2023-02-16T10:40:55Z)
• SIDER: Single-Image Neural Optimization for Facial Geometric Detail Recovery [54.64663713249079]
  SIDER is a novel photometric optimization method that recovers detailed facial geometry from a single image in an unsupervised manner. In contrast to prior work, SIDER does not rely on any dataset priors and does not require additional supervision from multiple views, lighting changes or ground truth 3D shape.
  arXiv  Detail & Related papers  (2021-08-11T22:34:53Z)
• Improving Metric Dimensionality Reduction with Distributed Topology [68.8204255655161]
  DIPOLE is a dimensionality-reduction post-processing step that corrects an initial embedding by minimizing a loss functional with both a local, metric term and a global, topological term. We observe that DIPOLE outperforms popular methods like UMAP, t-SNE, and Isomap on a number of popular datasets.
  arXiv  Detail & Related papers  (2021-06-14T17:19:44Z)
• Adaptive Surface Normal Constraint for Depth Estimation [102.7466374038784]
  We introduce a simple yet effective method, named Adaptive Surface Normal (ASN) constraint, to correlate the depth estimation with geometric consistency. Our method can faithfully reconstruct the 3D geometry and is robust to local shape variations, such as boundaries, sharp corners and noises.
  arXiv  Detail & Related papers  (2021-03-29T10:36:25Z)
• Self-supervised Geometric Perception [96.89966337518854]
  Self-supervised geometric perception is a framework to learn a feature descriptor for correspondence matching without any ground-truth geometric model labels. We show that SGP achieves state-of-the-art performance that is on-par or superior to the supervised oracles trained using ground-truth labels.
  arXiv  Detail & Related papers  (2021-03-04T15:34:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.