LDDMM-Face: Large Deformation Diffeomorphic Metric Learning for Flexible and Consistent Face Alignment

• URL: http://arxiv.org/abs/2108.00690v1
• Date: Mon, 2 Aug 2021 07:57:15 GMT
• Title: LDDMM-Face: Large Deformation Diffeomorphic Metric Learning for Flexible and Consistent Face Alignment
• Authors: Huilin Yang, Junyan Lyu, Pujin Cheng, Xiaoying Tang
• Abstract summary: We propose a flexible and consistent face alignment framework, LDDMM-Face. The key contribution is a deformation layer that naturally embeds facial geometry in a diffeomorphic way. We extensively evaluate LDDMM-Face on four benchmark datasets: 300W, WFLW, HELEN and COFW-68.
• Score: 0.745554610293091
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We innovatively propose a flexible and consistent face alignment framework, LDDMM-Face, the key contribution of which is a deformation layer that naturally embeds facial geometry in a diffeomorphic way. Instead of predicting facial landmarks via heatmap or coordinate regression, we formulate this task in a diffeomorphic registration manner and predict momenta that uniquely parameterize the deformation between initial boundary and true boundary, and then perform large deformation diffeomorphic metric mapping (LDDMM) simultaneously for curve and landmark to localize the facial landmarks. Due to the embedding of LDDMM into a deep network, LDDMM-Face can consistently annotate facial landmarks without ambiguity and flexibly handle various annotation schemes, and can even predict dense annotations from sparse ones. Our method can be easily integrated into various face alignment networks. We extensively evaluate LDDMM-Face on four benchmark datasets: 300W, WFLW, HELEN and COFW-68. LDDMM-Face is comparable or superior to state-of-the-art methods for traditional within-dataset and same-annotation settings, but truly distinguishes itself with outstanding performance when dealing with weakly-supervised learning (partial-to-full), challenging cases (e.g., occluded faces), and different training and prediction datasets. In addition, LDDMM-Face shows promising results on the most challenging task of predicting across datasets with different annotation schemes.

Related papers

• Attributes Shape the Embedding Space of Face Recognition Models [0.0]
  Face Recognition tasks have made significant progress with the advent of Deep Neural Networks.<n>We observe a multiscale geometric structure emerging in the embedding space.<n>We propose a geometric approach to describe the dependence or invariance of FR models to these attributes.
  arXiv  Detail & Related papers  (2025-07-15T14:44:39Z)
• Mitigating Knowledge Discrepancies among Multiple Datasets for Task-agnostic Unified Face Alignment [30.501432077729245]
  Despite the similar structures of human faces, existing face alignment methods cannot learn unified knowledge from multiple datasets. This paper presents a strategy to unify knowledge from multiple datasets. The successful mitigation of discrepancies also enhances the efficiency of knowledge transferring to a novel dataset.
  arXiv  Detail & Related papers  (2025-03-28T11:59:27Z)
• FineCLIPER: Multi-modal Fine-grained CLIP for Dynamic Facial Expression Recognition with AdaptERs [5.35588281968644]
  We propose a novel framework, named Multi-modal Fine-grained CLIP for Dynamic Facial Expression Recognition with AdaptERs (Fine CLIPER) Our Fine CLIPER achieves tunable SOTA performance on the DFEW, FERV39k, and MAFW datasets with few parameters.
  arXiv  Detail & Related papers  (2024-07-02T10:55:43Z)
• Towards Geometric-Photometric Joint Alignment for Facial Mesh Registration [3.1932242398896964]
  This paper presents a Geometric-Photometric Joint Alignment(GPJA) method. It aligns discrete human expressions at pixel-level accuracy by combining geometric and photometric information. This consistency benefits face animation, re-parametrization, and other batch operations for face modeling and applications with enhanced efficiency.
  arXiv  Detail & Related papers  (2024-03-05T03:39:23Z)
• Optimal-Landmark-Guided Image Blending for Face Morphing Attacks [8.024953195407502]
  We propose a novel approach for conducting face morphing attacks, which utilizes optimal-landmark-guided image blending. Our proposed method overcomes the limitations of previous approaches by optimizing the morphing landmarks and using Graph Convolutional Networks (GCNs) to combine landmark and appearance features.
  arXiv  Detail & Related papers  (2024-01-30T03:45:06Z)
• Learning Layer-wise Equivariances Automatically using Gradients [66.81218780702125]
  Convolutions encode equivariance symmetries into neural networks leading to better generalisation performance. symmetries provide fixed hard constraints on the functions a network can represent, need to be specified in advance, and can not be adapted. Our goal is to allow flexible symmetry constraints that can automatically be learned from data using gradients.
  arXiv  Detail & Related papers  (2023-10-09T20:22:43Z)
• BlendFields: Few-Shot Example-Driven Facial Modeling [35.86727715239676]
  We introduce a method that bridges the gap by drawing inspiration from traditional computer graphics techniques. Unseen expressions are modeled by blending appearance from a sparse set of extreme poses. We show that our method generalizes to unseen expressions, adding fine-grained effects on top of smooth volumetric deformations of a face, and demonstrate how it generalizes beyond faces.
  arXiv  Detail & Related papers  (2023-05-12T14:30:07Z)
• Decoupled Multi-task Learning with Cyclical Self-Regulation for Face Parsing [71.19528222206088]
  We propose a novel Decoupled Multi-task Learning with Cyclical Self-Regulation for face parsing. Specifically, DML-CSR designs a multi-task model which comprises face parsing, binary edge, and category edge detection. Our method achieves the new state-of-the-art performance on the Helen, CelebA-HQ, and LapaMask datasets.
  arXiv  Detail & Related papers  (2022-03-28T02:12:30Z)
• Implicit Neural Deformation for Multi-View Face Reconstruction [43.88676778013593]
  We present a new method for 3D face reconstruction from multi-view RGB images. Unlike previous methods which are built upon 3D morphable models, our method leverages an implicit representation to encode rich geometric features. Our experimental results on several benchmark datasets demonstrate that our approach outperforms alternative baselines and achieves superior face reconstruction results compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-12-05T07:02:53Z)
• Shape My Face: Registering 3D Face Scans by Surface-to-Surface Translation [75.59415852802958]
  Shape-My-Face (SMF) is a powerful encoder-decoder architecture based on an improved point cloud encoder, a novel visual attention mechanism, graph convolutional decoders with skip connections, and a specialized mouth model. Our model provides topologically-sound meshes with minimal supervision, offers faster training time, has orders of magnitude fewer trainable parameters, is more robust to noise, and can generalize to previously unseen datasets.
  arXiv  Detail & Related papers  (2020-12-16T20:02:36Z)
• BioMetricNet: deep unconstrained face verification through learning of metrics regularized onto Gaussian distributions [25.00475462213752]
  We present BioMetricNet, a novel framework for deep unconstrained face verification. The proposed approach does not impose any specific metric on facial features. It shapes the decision space by learning a latent representation in which matching and non-matching pairs are mapped onto clearly separated and well-behaved target distributions.
  arXiv  Detail & Related papers  (2020-08-13T17:22:46Z)
• Semi-Siamese Training for Shallow Face Learning [78.7386209619276]
  We introduce a novel training method named Semi-Siamese Training (SST) A pair of Semi-Siamese networks constitute the forward propagation structure, and the training loss is computed with an updating gallery queue. Our method is developed without extra-dependency, thus can be flexibly integrated with the existing loss functions and network architectures.
  arXiv  Detail & Related papers  (2020-07-16T15:20:04Z)
• Multi-Margin based Decorrelation Learning for Heterogeneous Face Recognition [90.26023388850771]
  This paper presents a deep neural network approach to extract decorrelation representations in a hyperspherical space for cross-domain face images. The proposed framework can be divided into two components: heterogeneous representation network and decorrelation representation learning. Experimental results on two challenging heterogeneous face databases show that our approach achieves superior performance on both verification and recognition tasks.
  arXiv  Detail & Related papers  (2020-05-25T07:01:12Z)
• Towards Universal Representation Learning for Deep Face Recognition [106.21744671876704]
  We propose a universal representation learning framework that can deal with larger variation unseen in the given training data without leveraging target domain knowledge. Experiments show that our method achieves top performance on general face recognition datasets such as LFW and MegaFace.
  arXiv  Detail & Related papers  (2020-02-26T23:29:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.