A Theory of Stabilization by Skull Carving

• URL: http://arxiv.org/abs/2411.05827v1
• Date: Tue, 05 Nov 2024 17:54:05 GMT
• Title: A Theory of Stabilization by Skull Carving
• Authors: Mathieu Lamarre, Patrick Anderson, Étienne Danvoye,
• Abstract summary: Accurate stabilization of facial motion is essential for applications in photoreal avatar construction for 3D games, virtual reality, movies, and training data collection. We introduce the concept of a stable hull as the surface of the intersection of stabilized scans, analogous to the visual hull in shape-from-silhouette and the photo hull from space carving.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Accurate stabilization of facial motion is essential for applications in photoreal avatar construction for 3D games, virtual reality, movies, and training data collection. For the latter, stabilization must work automatically for the general population with people of varying morphology. Distinguishing rigid skull motion from facial expressions is critical since misalignment between skull motion and facial expressions can lead to animation models that are hard to control and can not fit natural motion. Existing methods struggle to work with sparse sets of very different expressions, such as when combining multiple units from the Facial Action Coding System (FACS). Certain approaches are not robust enough, some depend on motion data to find stable points, while others make one-for-all invalid physiological assumptions. In this paper, we leverage recent advances in neural signed distance fields and differentiable isosurface meshing to compute skull stabilization rigid transforms directly on unstructured triangle meshes or point clouds, significantly enhancing accuracy and robustness. We introduce the concept of a stable hull as the surface of the boolean intersection of stabilized scans, analogous to the visual hull in shape-from-silhouette and the photo hull from space carving. This hull resembles a skull overlaid with minimal soft tissue thickness, upper teeth are automatically included. Our skull carving algorithm simultaneously optimizes the stable hull shape and rigid transforms to get accurate stabilization of complex expressions for large diverse sets of people, outperforming existing methods.

Related papers

• A Novel Evolutionary Method for Automated Skull-Face Overlay in Computer-Aided Craniofacial Superimposition [2.3880383038985458]
  Lilium is an evolutionary method to enhance the accuracy and robustness of skull-face overlay.<n>It enforces anatomical, morphological, and photographic plausibility through a combination of constraints.
  arXiv  Detail & Related papers  (2026-02-26T13:55:19Z)
• Make-It-Poseable: Feed-forward Latent Posing Model for 3D Humanoid Character Animation [74.6792422278706]
  We introduce Make-It-Poseable, a novel feed-forward framework that reformulates character posing as a latent-space transformation problem.<n>Our method reconstructs the character in new poses by directly manipulating its latent representation.<n>It also naturally extends to 3D editing applications like part replacement and refinement.
  arXiv  Detail & Related papers  (2025-12-18T17:01:44Z)
• DreamActor-M1: Holistic, Expressive and Robust Human Image Animation with Hybrid Guidance [9.898947423344884]
  We propose a diffusion transformer (DiT) based framework, DreamActor-M1, with hybrid guidance to overcome limitations. For motion guidance, our hybrid control signals that integrate implicit facial representations, 3D head spheres, and 3D body skeletons achieve robust control of facial expressions and body movements. Experiments demonstrate that our method outperforms the state-of-the-art works, delivering expressive results for portraits, upper-body, and full-body generation.
  arXiv  Detail & Related papers  (2025-04-02T13:30:32Z)
• One-shot Human Motion Transfer via Occlusion-Robust Flow Prediction and Neural Texturing [21.613055849276385]
  We propose a unified framework that combines multi-scale feature warping and neural texture mapping to recover better 2D appearance and 2.5D geometry. Our model takes advantage of multiple modalities by jointly training and fusing them, which allows it to robust neural texture features that cope with geometric errors.
  arXiv  Detail & Related papers  (2024-12-09T03:14:40Z)
• Learning to Stabilize Faces [19.862045104684636]
  We present a new learning-based approach that is simple and fully automatic. We generate synthetic training data using a 3D Morphable Model (3DMM), exploiting the fact that 3DMM parameters separate skull motion from facial skin motion. We show that our approach outperforms the state-of-the-art both quantitatively and qualitatively on the tasks of stabilizing discrete sets of facial expressions.
  arXiv  Detail & Related papers  (2024-11-22T17:03:25Z)
• Learning a Generalized Physical Face Model From Data [19.58920251013244]
  Physically-based simulation is a powerful approach for 3D facial animation. We propose a generalized physical face model that we learn from a large 3D face dataset. Our model can be quickly fit to any unseen identity and produce a ready-to-animate physical face model automatically.
  arXiv  Detail & Related papers  (2024-02-29T18:59:31Z)
• Decaf: Monocular Deformation Capture for Face and Hand Interactions [77.75726740605748]
  This paper introduces the first method that allows tracking human hands interacting with human faces in 3D from single monocular RGB videos. We model hands as articulated objects inducing non-rigid face deformations during an active interaction. Our method relies on a new hand-face motion and interaction capture dataset with realistic face deformations acquired with a markerless multi-view camera system.
  arXiv  Detail & Related papers  (2023-09-28T17:59:51Z)
• 3D shape reconstruction of semi-transparent worms [0.950214811819847]
  3D shape reconstruction typically requires identifying object features or textures in multiple images of a subject. Here we overcome these challenges by rendering a candidate shape with adaptive blurring and transparency for comparison with the images. We model the slender Caenorhabditis elegans as a 3D curve using an intrinsic parametrisation that naturally admits biologically-informed constraints and regularisation.
  arXiv  Detail & Related papers  (2023-04-28T13:29:36Z)
• MoDA: Modeling Deformable 3D Objects from Casual Videos [84.29654142118018]
  We propose neural dual quaternion blend skinning (NeuDBS) to achieve 3D point deformation without skin-collapsing artifacts. In the endeavor to register 2D pixels across different frames, we establish a correspondence between canonical feature embeddings that encodes 3D points within the canonical space. Our approach can reconstruct 3D models for humans and animals with better qualitative and quantitative performance than state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-04-17T13:49:04Z)
• Imposing Temporal Consistency on Deep Monocular Body Shape and Pose Estimation [67.23327074124855]
  This paper presents an elegant solution for the integration of temporal constraints in the fitting process. We derive parameters of a sequence of body models, representing shape and motion of a person, including jaw poses, facial expressions, and finger poses. Our approach enables the derivation of realistic 3D body models from image sequences, including facial expression and articulated hands.
  arXiv  Detail & Related papers  (2022-02-07T11:11:55Z)
• Detailed Avatar Recovery from Single Image [50.82102098057822]
  This paper presents a novel framework to recover emphdetailed avatar from a single image. We use the deep neural networks to refine the 3D shape in a Hierarchical Mesh Deformation framework. Our method can restore detailed human body shapes with complete textures beyond skinned models.
  arXiv  Detail & Related papers  (2021-08-06T03:51:26Z)
• Learning Skeletal Articulations with Neural Blend Shapes [57.879030623284216]
  We develop a neural technique for articulating 3D characters using enveloping with a pre-defined skeletal structure. Our framework learns to rig and skin characters with the same articulation structure. We propose neural blend shapes which improve the deformation quality in the joint regions.
  arXiv  Detail & Related papers  (2021-05-06T05:58:13Z)
• SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes [117.76767853430243]
  We introduce SNARF, which combines the advantages of linear blend skinning for polygonal meshes with neural implicit surfaces. We propose a forward skinning model that finds all canonical correspondences of any deformed point using iterative root finding. Compared to state-of-the-art neural implicit representations, our approach generalizes better to unseen poses while preserving accuracy.
  arXiv  Detail & Related papers  (2021-04-08T17:54:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.