Robust 3D Face Alignment with Multi-Path Neural Architecture Search

• URL: http://arxiv.org/abs/2406.07873v1
• Date: Wed, 12 Jun 2024 05:02:16 GMT
• Title: Robust 3D Face Alignment with Multi-Path Neural Architecture Search
• Authors: Zhichao Jiang, Hongsong Wang, Xi Teng, Baopu Li,
• Abstract summary: 3D face alignment is a very challenging and fundamental problem in computer vision. Existing deep learning-based methods manually design different networks to regress either parameters of a 3D face model or 3D positions of face vertices. We employ Neural Architecture Search (NAS) to automatically discover the optimal architecture for 3D face alignment.
• Score: 23.432737053236096
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: 3D face alignment is a very challenging and fundamental problem in computer vision. Existing deep learning-based methods manually design different networks to regress either parameters of a 3D face model or 3D positions of face vertices. However, designing such networks relies on expert knowledge, and these methods often struggle to produce consistent results across various face poses. To address this limitation, we employ Neural Architecture Search (NAS) to automatically discover the optimal architecture for 3D face alignment. We propose a novel Multi-path One-shot Neural Architecture Search (MONAS) framework that leverages multi-scale features and contextual information to enhance face alignment across various poses. The MONAS comprises two key algorithms: Multi-path Networks Unbiased Sampling Based Training and Simulated Annealing based Multi-path One-shot Search. Experimental results on three popular benchmarks demonstrate the superior performance of the MONAS for both sparse alignment and dense alignment.

Related papers

• PointSeg: A Training-Free Paradigm for 3D Scene Segmentation via Foundation Models [51.24979014650188]
  We present PointSeg, a training-free paradigm that leverages off-the-shelf vision foundation models to address 3D scene perception tasks. PointSeg can segment anything in 3D scene by acquiring accurate 3D prompts to align their corresponding pixels across frames. Our approach significantly surpasses the state-of-the-art specialist training-free model by 14.1$%$, 12.3$%$, and 12.6$%$ mAP on ScanNet, ScanNet++, and KITTI-360 datasets.
  arXiv  Detail & Related papers  (2024-03-11T03:28:20Z)
• Wonder3D: Single Image to 3D using Cross-Domain Diffusion [105.16622018766236]
  Wonder3D is a novel method for efficiently generating high-fidelity textured meshes from single-view images. To holistically improve the quality, consistency, and efficiency of image-to-3D tasks, we propose a cross-domain diffusion model.
  arXiv  Detail & Related papers  (2023-10-23T15:02:23Z)
• SeMLaPS: Real-time Semantic Mapping with Latent Prior Networks and Quasi-Planar Segmentation [53.83313235792596]
  We present a new methodology for real-time semantic mapping from RGB-D sequences. It combines a 2D neural network and a 3D network based on a SLAM system with 3D occupancy mapping. Our system achieves state-of-the-art semantic mapping quality within 2D-3D networks-based systems.
  arXiv  Detail & Related papers  (2023-06-28T22:36:44Z)
• 3D Dense Face Alignment with Fused Features by Aggregating CNNs and GCNs [28.7443367565456]
  This is achieved by seamlessly combining standard convolutional neural networks (CNNs) with Graph Convolution Networks (GCNs) By iteratively fusing the features across different layers and stages of the CNNs and GCNs, our approach can provide a dense face alignment and 3D face reconstruction simultaneously. Experiments on several challenging datasets demonstrate that our method outperforms state-of-the-art approaches on both 2D and 3D face alignment tasks.
  arXiv  Detail & Related papers  (2022-03-09T11:07:10Z)
• Multi-initialization Optimization Network for Accurate 3D Human Pose and Shape Estimation [75.44912541912252]
  We propose a three-stage framework named Multi-Initialization Optimization Network (MION) In the first stage, we strategically select different coarse 3D reconstruction candidates which are compatible with the 2D keypoints of input sample. In the second stage, we design a mesh refinement transformer (MRT) to respectively refine each coarse reconstruction result via a self-attention mechanism. Finally, a Consistency Estimation Network (CEN) is proposed to find the best result from mutiple candidates by evaluating if the visual evidence in RGB image matches a given 3D reconstruction.
  arXiv  Detail & Related papers  (2021-12-24T02:43:58Z)
• Implicit Neural Deformation for Multi-View Face Reconstruction [43.88676778013593]
  We present a new method for 3D face reconstruction from multi-view RGB images. Unlike previous methods which are built upon 3D morphable models, our method leverages an implicit representation to encode rich geometric features. Our experimental results on several benchmark datasets demonstrate that our approach outperforms alternative baselines and achieves superior face reconstruction results compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-12-05T07:02:53Z)
• VolumeFusion: Deep Depth Fusion for 3D Scene Reconstruction [71.83308989022635]
  In this paper, we advocate that replicating the traditional two stages framework with deep neural networks improves both the interpretability and the accuracy of the results. Our network operates in two steps: 1) the local computation of the local depth maps with a deep MVS technique, and, 2) the depth maps and images' features fusion to build a single TSDF volume. In order to improve the matching performance between images acquired from very different viewpoints, we introduce a rotation-invariant 3D convolution kernel called PosedConv.
  arXiv  Detail & Related papers  (2021-08-19T11:33:58Z)
• Accurate 3D Facial Geometry Prediction by Multi-Task, Multi-Modal, and Multi-Representation Landmark Refinement Network [14.966695101335704]
  This work focuses on complete 3D facial geometry prediction, including 3D facial alignment via 3D modeling and face orientation estimation. Our focus is on the important facial attributes, 3D landmarks, and we fully utilize their embedded information to guide 3D facial geometry learning. We attain the state of the art from extensive experiments on all tasks of learning 3D facial geometry.
  arXiv  Detail & Related papers  (2021-04-16T23:22:41Z)
• Learning 3D Face Reconstruction with a Pose Guidance Network [49.13404714366933]
  We present a self-supervised learning approach to learning monocular 3D face reconstruction with a pose guidance network (PGN) First, we unveil the bottleneck of pose estimation in prior parametric 3D face learning methods, and propose to utilize 3D face landmarks for estimating pose parameters. With our specially designed PGN, our model can learn from both faces with fully labeled 3D landmarks and unlimited unlabeled in-the-wild face images.
  arXiv  Detail & Related papers  (2020-10-09T06:11:17Z)
• Self-Supervised Monocular 3D Face Reconstruction by Occlusion-Aware Multi-view Geometry Consistency [40.56510679634943]
  We propose a self-supervised training architecture by leveraging the multi-view geometry consistency. We design three novel loss functions for multi-view consistency, including the pixel consistency loss, the depth consistency loss, and the facial landmark-based epipolar loss. Our method is accurate and robust, especially under large variations of expressions, poses, and illumination conditions.
  arXiv  Detail & Related papers  (2020-07-24T12:36:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.