Polarimetric BSSRDF Acquisition of Dynamic Faces

• URL: http://arxiv.org/abs/2501.01980v1
• Date: Sun, 29 Dec 2024 07:57:25 GMT
• Title: Polarimetric BSSRDF Acquisition of Dynamic Faces
• Authors: Hyunho Ha, Inseung Hwang, Nestor Monzon, Jaemin Cho, Donggun Kim, Seung-Hwan Baek, Adolfo Muñoz, Diego Gutierrez, Min H. Kim,
• Abstract summary: We present a new polarimetric acquisition method for dynamic human faces. It captures spatially varying appearance and precise geometry across a wide spectrum of skin tones and facial expressions. Our work is the first to simultaneously acquire polarimetric and spectral reflectance information alongside biophysically-based skin parameters and geometry.
• Score: 29.92829310793318
• License:
• Abstract: Acquisition and modeling of polarized light reflection and scattering help reveal the shape, structure, and physical characteristics of an object, which is increasingly important in computer graphics. However, current polarimetric acquisition systems are limited to static and opaque objects. Human faces, on the other hand, present a particularly difficult challenge, given their complex structure and reflectance properties, the strong presence of spatially-varying subsurface scattering, and their dynamic nature. We present a new polarimetric acquisition method for dynamic human faces, which focuses on capturing spatially varying appearance and precise geometry, across a wide spectrum of skin tones and facial expressions. It includes both single and heterogeneous subsurface scattering, index of refraction, and specular roughness and intensity, among other parameters, while revealing biophysically-based components such as inner- and outer-layer hemoglobin, eumelanin and pheomelanin. Our method leverages such components' unique multispectral absorption profiles to quantify their concentrations, which in turn inform our model about the complex interactions occurring within the skin layers. To our knowledge, our work is the first to simultaneously acquire polarimetric and spectral reflectance information alongside biophysically-based skin parameters and geometry of dynamic human faces. Moreover, our polarimetric skin model integrates seamlessly into various rendering pipelines.

Related papers

• Geometric Trajectory Diffusion Models [58.853975433383326]
  Generative models have shown great promise in generating 3D geometric systems. Existing approaches only operate on static structures, neglecting the fact that physical systems are always dynamic in nature. We propose geometric trajectory diffusion models (GeoTDM), the first diffusion model for modeling the temporal distribution of 3D geometric trajectories.
  arXiv  Detail & Related papers  (2024-10-16T20:36:41Z)
• NeRSP: Neural 3D Reconstruction for Reflective Objects with Sparse Polarized Images [62.752710734332894]
  NeRSP is a Neural 3D reconstruction technique for Reflective surfaces with Sparse Polarized images. We derive photometric and geometric cues from the polarimetric image formation model and multiview azimuth consistency. We achieve the state-of-the-art surface reconstruction results with only 6 views as input.
  arXiv  Detail & Related papers  (2024-06-11T09:53:18Z)
• Polaritons under Extensive Disordered Molecular Rotation in Optical Cavities [4.788427041690547]
  This study investigates the dynamic behavior of polaritons in an optical cavity containing one million molecules. The rotational motion of molecules significantly affects the electromagnetic field distribution within the cavity. The presence of level disorder induces diverse energy level structures, influencing the energy distribution of polaritons.
  arXiv  Detail & Related papers  (2023-12-28T08:31:53Z)
• S2P3: Self-Supervised Polarimetric Pose Prediction [55.43547228561919]
  This paper proposes the first self-supervised 6D object pose prediction from multimodal RGB+polarimetric images. The novel training paradigm comprises 1) a physical model to extract geometric information of polarized light, 2) a teacher-student knowledge distillation scheme and 3) a self-supervised loss formulation through differentiable constraints.
  arXiv  Detail & Related papers  (2023-12-02T10:46:40Z)
• NePF: Neural Photon Field for Single-Stage Inverse Rendering [6.977356702921476]
  We present a novel single-stage framework, Neural Photon Field (NePF), to address the ill-posed inverse rendering from multi-view images. NePF achieves this unification by fully utilizing the physical implication behind the weight function of neural implicit surfaces. We evaluate our method on both real and synthetic datasets.
  arXiv  Detail & Related papers  (2023-11-20T06:15:46Z)
• MEGANE: Morphable Eyeglass and Avatar Network [83.65790119755053]
  We propose a 3D compositional morphable model of eyeglasses. We employ a hybrid representation that combines surface geometry and a volumetric representation. Our approach models global light transport effects, such as casting shadows between faces and glasses.
  arXiv  Detail & Related papers  (2023-02-09T18:59:49Z)
• Wave-packet and entanglement dynamics in a non-Hermitian many-body system [0.0]
  A one-dimensional quantum system described by a non-Hermitian Hamiltonian of the so-called Hatano-Nelson type is studied. Effects of disorder and inter-particle interaction, especially, when they coexist, may be less understood.
  arXiv  Detail & Related papers  (2022-12-02T06:47:18Z)
• Generalizable Neural Performer: Learning Robust Radiance Fields for Human Novel View Synthesis [52.720314035084215]
  This work targets at using a general deep learning framework to synthesize free-viewpoint images of arbitrary human performers. We present a simple yet powerful framework, named Generalizable Neural Performer (GNR), that learns a generalizable and robust neural body representation. Experiments on GeneBody-1.0 and ZJU-Mocap show better robustness of our methods than recent state-of-the-art generalizable methods.
  arXiv  Detail & Related papers  (2022-04-25T17:14:22Z)
• Deep Physics-aware Inference of Cloth Deformation for Monocular Human Performance Capture [84.73946704272113]
  We show how integrating physics into the training process improves the learned cloth deformations and allows modeling clothing as a separate piece of geometry. Our approach leads to a significant improvement over current state-of-the-art methods and is thus a clear step towards realistic monocular capture of the entire deforming surface of a human clothed.
  arXiv  Detail & Related papers  (2020-11-25T16:46:00Z)
• Learning Formation of Physically-Based Face Attributes [16.55993873730069]
  Based on a combined data set of 4000 high resolution facial scans, we introduce a non-linear morphable face model. Our deep learning based generative model learns to correlate albedo and geometry, which ensures the anatomical correctness of the generated assets.
  arXiv  Detail & Related papers  (2020-04-02T07:01:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.