BecomingLit: Relightable Gaussian Avatars with Hybrid Neural Shading

• URL: http://arxiv.org/abs/2506.06271v1
• Date: Fri, 06 Jun 2025 17:53:58 GMT
• Title: BecomingLit: Relightable Gaussian Avatars with Hybrid Neural Shading
• Authors: Jonathan Schmidt, Simon Giebenhain, Matthias Niessner,
• Abstract summary: We introduce BecomingLit, a novel method for reconstructing relightable, high-resolution head avatars that can be rendered from novel viewpoints at interactive rates.<n>We collect a novel dataset consisting of diverse multi-view sequences of numerous subjects under varying illumination conditions.<n>We propose a new hybrid neural shading approach, combining a neural diffuse BRDF with an analytical specular term.
• Score: 3.447848701446988
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce BecomingLit, a novel method for reconstructing relightable, high-resolution head avatars that can be rendered from novel viewpoints at interactive rates. Therefore, we propose a new low-cost light stage capture setup, tailored specifically towards capturing faces. Using this setup, we collect a novel dataset consisting of diverse multi-view sequences of numerous subjects under varying illumination conditions and facial expressions. By leveraging our new dataset, we introduce a new relightable avatar representation based on 3D Gaussian primitives that we animate with a parametric head model and an expression-dependent dynamics module. We propose a new hybrid neural shading approach, combining a neural diffuse BRDF with an analytical specular term. Our method reconstructs disentangled materials from our dynamic light stage recordings and enables all-frequency relighting of our avatars with both point lights and environment maps. In addition, our avatars can easily be animated and controlled from monocular videos. We validate our approach in extensive experiments on our dataset, where we consistently outperform existing state-of-the-art methods in relighting and reenactment by a significant margin.

Related papers

• HoliGS: Holistic Gaussian Splatting for Embodied View Synthesis [59.25751939710903]
  We propose a novel deformable Gaussian splatting framework that addresses embodied view synthesis from long monocular RGB videos.<n>Our method leverages invertible Gaussian Splatting deformation networks to reconstruct large-scale, dynamic environments accurately.<n>Results highlight a practical and scalable solution for EVS in real-world scenarios.
  arXiv  Detail & Related papers  (2025-06-24T03:54:40Z)
• URAvatar: Universal Relightable Gaussian Codec Avatars [42.25313535192927]
  We present a new approach to creating photorealistic and relightable head avatars from a phone scan with unknown illumination. The reconstructed avatars can be animated and relit in real time with the global illumination of diverse environments.
  arXiv  Detail & Related papers  (2024-10-31T17:59:56Z)
• Interactive Rendering of Relightable and Animatable Gaussian Avatars [37.73483372890271]
  We propose a simple and efficient method to decouple body materials and lighting from multi-view or monocular avatar videos.<n>Our method can render higher quality results at a faster speed on both synthetic and real datasets.
  arXiv  Detail & Related papers  (2024-07-15T13:25:07Z)
• MetaGS: A Meta-Learned Gaussian-Phong Model for Out-of-Distribution 3D Scene Relighting [63.5925701087252]
  Out-of-distribution (OOD) 3D relighting requires novel view synthesis under unseen lighting conditions.<n>We introduce MetaGS to tackle this challenge from two perspectives.
  arXiv  Detail & Related papers  (2024-05-31T13:48:54Z)
• Relightable Neural Actor with Intrinsic Decomposition and Pose Control [80.06094206522668]
  We propose Relightable Neural Actor, a new video-based method for learning a pose-driven neural human model that can be relighted. For training, our method solely requires a multi-view recording of the human under a known, but static lighting condition. To evaluate our approach in real-world scenarios, we collect a new dataset with four identities recorded under different light conditions, indoors and outdoors.
  arXiv  Detail & Related papers  (2023-12-18T14:30:13Z)
• Relightable Gaussian Codec Avatars [26.255161061306428]
  We present Relightable Gaussian Codec Avatars, a method to build high-fidelity relightable head avatars that can be animated to generate novel expressions. Our geometry model based on 3D Gaussians can capture 3D-consistent sub-millimeter details such as hair strands and pores on dynamic face sequences. We improve the fidelity of eye reflections and enable explicit gaze control by introducing relightable explicit eye models.
  arXiv  Detail & Related papers  (2023-12-06T18:59:58Z)
• Artist-Friendly Relightable and Animatable Neural Heads [15.803111500220888]
  A common approach for creating photo-realistic digital avatars is through the use of volumetric neural fields. Recent variants also surpassed the usual drawback of baked-in illumination in neural representations, showing that static neural avatars can be relit in any environment. Our method builds on a proven dynamic avatar approach based on a mixture of volumetric primitives, combined with a recently-proposed lightweight hardware setup for relightable neural fields.
  arXiv  Detail & Related papers  (2023-12-06T11:06:46Z)
• FLARE: Fast Learning of Animatable and Relightable Mesh Avatars [64.48254296523977]
  Our goal is to efficiently learn personalized animatable 3D head avatars from videos that are geometrically accurate, realistic, relightable, and compatible with current rendering systems. We introduce FLARE, a technique that enables the creation of animatable and relightable avatars from a single monocular video.
  arXiv  Detail & Related papers  (2023-10-26T16:13:00Z)
• Towards Practical Capture of High-Fidelity Relightable Avatars [60.25823986199208]
  TRAvatar is trained with dynamic image sequences captured in a Light Stage under varying lighting conditions. It can predict the appearance in real-time with a single forward pass, achieving high-quality relighting effects. Our framework achieves superior performance for photorealistic avatar animation and relighting.
  arXiv  Detail & Related papers  (2023-09-08T10:26:29Z)
• TensoIR: Tensorial Inverse Rendering [51.57268311847087]
  TensoIR is a novel inverse rendering approach based on tensor factorization and neural fields. TensoRF is a state-of-the-art approach for radiance field modeling.
  arXiv  Detail & Related papers  (2023-04-24T21:39:13Z)
• Neural 3D Reconstruction in the Wild [86.6264706256377]
  We introduce a new method that enables efficient and accurate surface reconstruction from Internet photo collections. We present a new benchmark and protocol for evaluating reconstruction performance on such in-the-wild scenes.
  arXiv  Detail & Related papers  (2022-05-25T17:59:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.