Towards Fine-grained 3D Face Dense Registration: An Optimal Dividing and Diffusing Method

• URL: http://arxiv.org/abs/2109.11204v1
• Date: Thu, 23 Sep 2021 08:31:35 GMT
• Title: Towards Fine-grained 3D Face Dense Registration: An Optimal Dividing and Diffusing Method
• Authors: Zhenfeng Fan, Silong Peng, Shihong Xia
• Abstract summary: Dense-to-vertex correspondence between 3D faces is a fundamental and challenging issue for 3D&2D face analysis. In this paper, we revisit dense registration by a dimension-degraded problem, i.e. proportional segmentation of a line. We employ an iterative dividing and diffusing method to reach the final solution uniquely.
• Score: 17.38748022631488
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dense vertex-to-vertex correspondence between 3D faces is a fundamental and challenging issue for 3D&2D face analysis. While the sparse landmarks have anatomically ground-truth correspondence, the dense vertex correspondences on most facial regions are unknown. In this view, the current literatures commonly result in reasonable but diverse solutions, which deviate from the optimum to the 3D face dense registration problem. In this paper, we revisit dense registration by a dimension-degraded problem, i.e. proportional segmentation of a line, and employ an iterative dividing and diffusing method to reach the final solution uniquely. This method is then extended to 3D surface by formulating a local registration problem for dividing and a linear least-square problem for diffusing, with constraints on fixed features. On this basis, we further propose a multi-resolution algorithm to accelerate the computational process. The proposed method is linked to a novel local scaling metric, where we illustrate the physical meaning as smooth rearrangement for local cells of 3D facial shapes. Extensive experiments on public datasets demonstrate the effectiveness of the proposed method in various aspects. Generally, the proposed method leads to coherent local registrations and elegant mesh grid routines for fine-grained 3D face dense registrations, which benefits many downstream applications significantly. It can also be applied to dense correspondence for other format of data which are not limited to face. The core code will be publicly available at https://github.com/NaughtyZZ/3D_face_dense_registration.

Related papers

• Oriented-grid Encoder for 3D Implicit Representations [10.02138130221506]
  This paper is the first to exploit 3D characteristics in 3D geometric encoders explicitly. Our method gets state-of-the-art results when compared to the prior techniques.
  arXiv  Detail & Related papers  (2024-02-09T19:28:13Z)
• Unpaired Multi-domain Attribute Translation of 3D Facial Shapes with a Square and Symmetric Geometric Map [23.461476902880584]
  We propose a learning framework for 3D facial attribute translation. We use a novel geometric map for 3D shape representation and embed it in an end-to-end generative adversarial network. We employ a unified and unpaired learning framework for multi-domain attribute translation.
  arXiv  Detail & Related papers  (2023-08-25T08:37:55Z)
• Quadric Representations for LiDAR Odometry, Mapping and Localization [93.24140840537912]
  Current LiDAR odometry, mapping and localization methods leverage point-wise representations of 3D scenes. We propose a novel method of describing scenes using quadric surfaces, which are far more compact representations of 3D objects. Our method maintains low latency and memory utility while achieving competitive, and even superior, accuracy.
  arXiv  Detail & Related papers  (2023-04-27T13:52:01Z)
• Neural 3D Scene Reconstruction with the Manhattan-world Assumption [58.90559966227361]
  This paper addresses the challenge of reconstructing 3D indoor scenes from multi-view images. Planar constraints can be conveniently integrated into the recent implicit neural representation-based reconstruction methods. The proposed method outperforms previous methods by a large margin on 3D reconstruction quality.
  arXiv  Detail & Related papers  (2022-05-05T17:59:55Z)
• Topologically Consistent Multi-View Face Inference Using Volumetric Sampling [25.001398662643986]
  ToFu is a geometry inference framework that can produce topologically consistent meshes across identities and expressions. A novel progressive mesh generation network embeds the topological structure of the face in a feature volume. These high-quality assets are readily usable by production studios for avatar creation, animation and physically-based skin rendering.
  arXiv  Detail & Related papers  (2021-10-06T17:55:08Z)
• Smooth Mesh Estimation from Depth Data using Non-Smooth Convex Optimization [28.786685021545622]
  We build a 3D mesh directly from a depth map and the sparse landmarks triangulated with visual odometry. Our approach generates a smooth and accurate 3D mesh that substantially improves the state-of-the-art on direct mesh reconstruction while running in real-time.
  arXiv  Detail & Related papers  (2021-08-06T06:29:34Z)
• DeepMesh: Differentiable Iso-Surface Extraction [53.77622255726208]
  We introduce a differentiable way to produce explicit surface mesh representations from Deep Implicit Fields. Our key insight is that by reasoning on how implicit field perturbations impact local surface geometry, one can ultimately differentiate the 3D location of surface samples. We exploit this to define DeepMesh -- end-to-end differentiable mesh representation that can vary its topology.
  arXiv  Detail & Related papers  (2021-06-20T20:12:41Z)
• Deep Implicit Surface Point Prediction Networks [49.286550880464866]
  Deep neural representations of 3D shapes as implicit functions have been shown to produce high fidelity models. This paper presents a novel approach that models such surfaces using a new class of implicit representations called the closest surface-point (CSP) representation.
  arXiv  Detail & Related papers  (2021-06-10T14:31:54Z)
• Canny-VO: Visual Odometry with RGB-D Cameras based on Geometric 3D-2D Edge Alignment [85.32080531133799]
  This paper reviews the classical problem of free-form curve registration and applies it to an efficient RGBD visual odometry system called Canny-VO. Two replacements for the distance transformation commonly used in edge registration are proposed: Approximate Nearest Neighbour Fields and Oriented Nearest Neighbour Fields. 3D2D edge alignment benefits from these alternative formulations in terms of both efficiency and accuracy.
  arXiv  Detail & Related papers  (2020-12-15T11:42:17Z)
• Mapping of Sparse 3D Data using Alternating Projection [35.735398244213584]
  We propose a novel technique to register sparse 3D scans in the absence of texture. Existing methods such as KinectFusion heavily rely on dense point clouds. We propose the use of a two-step alternating projection algorithm by formulating the registration as the simultaneous satisfaction of intersection and rigidity constraints.
  arXiv  Detail & Related papers  (2020-10-04T17:40:30Z)
• Dense Non-Rigid Structure from Motion: A Manifold Viewpoint [162.88686222340962]
  Non-Rigid Structure-from-Motion (NRSfM) problem aims to recover 3D geometry of a deforming object from its 2D feature correspondences across multiple frames. We show that our approach significantly improves accuracy, scalability, and robustness against noise.
  arXiv  Detail & Related papers  (2020-06-15T09:15:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.