Multiple View Neural Regression of a Facial Shape Model

• URL: http://arxiv.org/abs/2608.22655v1
• Date: Sun, 23 Aug 2026 23:31:44 GMT
• Title: Multiple View Neural Regression of a Facial Shape Model
• Abstract summary: VarIS is a custom light sphere for capturing high-resolution stereo geometry and reflectance maps.<n>A deep learning framework is proposed to directly predict re-topologized facial meshes from synthetic multiview images.<n>The system produces standardized meshes ready for rigging and animation with minimal human supervision.
• Score: 5.202950948929751
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Creating re-topologized 3D facial meshes is essential for high-quality facial animation but remains labor-intensive and time-consuming. This dissertation explores more efficient approaches for capturing production-ready facial meshes through: (1) the development of VarIS, a custom light sphere for capturing high-resolution stereo geometry and reflectance maps; (2) analysis of camera parameters affecting automatic 2D and 3D landmarking; (3) synthetic-data methods for training neural face regression; and (4) techniques for improving neural multi-view face-shape regression. While VarIS enables photorealistic face capture, its operational and processing costs motivate a more scalable approach. A deep learning framework is therefore proposed to directly predict re-topologized facial meshes from synthetic multiview images generated with Visage Craft, an in-house physically based rendering system using an Appearance 3D Morphable Model (A3DMM). The system produces standardized meshes ready for rigging and animation with minimal human supervision. Results show that incorporating accurate camera intrinsics and extrinsics improves landmark accuracy and geometric consistency, while 3D landmark regularization further improves reconstruction quality.

Related papers

• GSwap: Realistic Head Swapping with Dynamic Neural Gaussian Field [66.31636070843516]
  GSwap is a consistent and realistic video head-swapping system empowered by dynamic neural Gaussian portrait priors.<n>GSwap surpasses existing methods in multiple aspects, including visual quality, temporal coherence, identity preservation, and 3D consistency.
  arXiv  Detail & Related papers  (2026-03-24T13:11:25Z)
• LHM: Large Animatable Human Reconstruction Model from a Single Image in Seconds [21.99354901986186]
  We propose LHM (Large Animatable Human Reconstruction Model) to infer high-fidelity avatars represented as 3D Gaussian splatting in a feed-forward pass.<n>Our model leverages a multimodal transformer architecture to effectively encode the human body positional features and image features with attention mechanism.<n>Our LHM generates plausible animatable human in seconds without post-processing for face and hands, outperforming existing methods in both reconstruction accuracy and generalization ability.
  arXiv  Detail & Related papers  (2025-03-13T17:59:21Z)
• SPARK: Self-supervised Personalized Real-time Monocular Face Capture [6.093606972415841]
  Current state of the art approaches have the ability to regress parametric 3D face models in real-time across a wide range of identities. We propose a method for high-precision 3D face capture taking advantage of a collection of unconstrained videos of a subject as prior information.
  arXiv  Detail & Related papers  (2024-09-12T12:30:04Z)
• SMIRK: 3D Facial Expressions through Analysis-by-Neural-Synthesis [30.2749903946587]
  SMIRK (Spatial Modeling for Image-based Reconstruction of Kinesics) faithfully reconstructs expressive 3D faces from images.<n>We identify two key limitations in existing methods: shortcomings in their self-supervised training formulation, and a lack of expression diversity in the training images.<n>Our qualitative, quantitative and particularly our perceptual evaluations demonstrate that SMIRK achieves the new state-of-the art performance on accurate expression reconstruction.
  arXiv  Detail & Related papers  (2024-04-05T14:00:07Z)
• Neural Point-based Volumetric Avatar: Surface-guided Neural Points for Efficient and Photorealistic Volumetric Head Avatar [62.87222308616711]
  We propose fullname (name), a method that adopts the neural point representation and the neural volume rendering process. Specifically, the neural points are strategically constrained around the surface of the target expression via a high-resolution UV displacement map. By design, our name is better equipped to handle topologically changing regions and thin structures while also ensuring accurate expression control when animating avatars.
  arXiv  Detail & Related papers  (2023-07-11T03:40:10Z)
• Video-driven Neural Physically-based Facial Asset for Production [33.24654834163312]
  We present a new learning-based, video-driven approach for generating dynamic facial geometries with high-quality physically-based assets. Our technique provides higher accuracy and visual fidelity than previous video-driven facial reconstruction and animation methods.
  arXiv  Detail & Related papers  (2022-02-11T13:22:48Z)
• AvatarMe++: Facial Shape and BRDF Inference with Photorealistic Rendering-Aware GANs [119.23922747230193]
  We introduce the first method that is able to reconstruct render-ready 3D facial geometry and BRDF from a single "in-the-wild" image. Our method outperforms the existing arts by a significant margin and reconstructs high-resolution 3D faces from a single low-resolution image.
  arXiv  Detail & Related papers  (2021-12-11T11:36:30Z)
• Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstruction [76.1612334630256]
  We harness the power of Generative Adversarial Networks (GANs) and Deep Convolutional Neural Networks (DCNNs) to reconstruct the facial texture and shape from single images. We demonstrate excellent results in photorealistic and identity preserving 3D face reconstructions and achieve for the first time, facial texture reconstruction with high-frequency details.
  arXiv  Detail & Related papers  (2021-05-16T16:35:44Z)
• Inverting Generative Adversarial Renderer for Face Reconstruction [58.45125455811038]
  In this work, we introduce a novel Generative Adversa Renderer (GAR) GAR learns to model the complicated real-world image, instead of relying on the graphics rules, it is capable of producing realistic images. Our method achieves state-of-the-art performances on multiple face reconstruction.
  arXiv  Detail & Related papers  (2021-05-06T04:16:06Z)
• Neural Descent for Visual 3D Human Pose and Shape [67.01050349629053]
  We present deep neural network methodology to reconstruct the 3d pose and shape of people, given an input RGB image. We rely on a recently introduced, expressivefull body statistical 3d human model, GHUM, trained end-to-end. Central to our methodology, is a learning to learn and optimize approach, referred to as HUmanNeural Descent (HUND), which avoids both second-order differentiation.
  arXiv  Detail & Related papers  (2020-08-16T13:38:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.