Relightable and Dynamic Gaussian Avatar Reconstruction from Monocular Video

• URL: http://arxiv.org/abs/2512.09335v2
• Date: Thu, 11 Dec 2025 04:18:41 GMT
• Title: Relightable and Dynamic Gaussian Avatar Reconstruction from Monocular Video
• Authors: Seonghwa Choi, Moonkyeong Choi, Mingyu Jang, Jaekyung Kim, Jianfei Cai, Wen-Huang Cheng, Sanghoon Lee,
• Abstract summary: We propose a 3DGS-based human avatar modeling framework, termed as Relightable and Dynamic Gaussian Avatar (RnD-Avatar)<n>Our framework enables realistic rendering of novel poses and views while supporting photo-realistic lighting effects under arbitrary lighting conditions.
• Score: 42.36677003378551
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modeling relightable and animatable human avatars from monocular video is a long-standing and challenging task. Recently, Neural Radiance Field (NeRF) and 3D Gaussian Splatting (3DGS) methods have been employed to reconstruct the avatars. However, they often produce unsatisfactory photo-realistic results because of insufficient geometrical details related to body motion, such as clothing wrinkles. In this paper, we propose a 3DGS-based human avatar modeling framework, termed as Relightable and Dynamic Gaussian Avatar (RnD-Avatar), that presents accurate pose-variant deformation for high-fidelity geometrical details. To achieve this, we introduce dynamic skinning weights that define the human avatar's articulation based on pose while also learning additional deformations induced by body motion. We also introduce a novel regularization to capture fine geometric details under sparse visual cues. Furthermore, we present a new multi-view dataset with varied lighting conditions to evaluate relight. Our framework enables realistic rendering of novel poses and views while supporting photo-realistic lighting effects under arbitrary lighting conditions. Our method achieves state-of-the-art performance in novel view synthesis, novel pose rendering, and relighting.

Related papers

• BecomingLit: Relightable Gaussian Avatars with Hybrid Neural Shading [3.447848701446988]
  We introduce BecomingLit, a novel method for reconstructing relightable, high-resolution head avatars that can be rendered from novel viewpoints at interactive rates.<n>We collect a novel dataset consisting of diverse multi-view sequences of numerous subjects under varying illumination conditions.<n>We propose a new hybrid neural shading approach, combining a neural diffuse BRDF with an analytical specular term.
  arXiv  Detail & Related papers  (2025-06-06T17:53:58Z)
• TeGA: Texture Space Gaussian Avatars for High-Resolution Dynamic Head Modeling [52.87836237427514]
  Photoreal avatars are seen as a key component in emerging applications in telepresence, extended reality, and entertainment.<n>We present a new high-detail 3D head avatar model that improves upon the state of the art.
  arXiv  Detail & Related papers  (2025-05-08T22:10:27Z)
• RealityAvatar: Towards Realistic Loose Clothing Modeling in Animatable 3D Gaussian Avatars [4.332718737928592]
  We propose RealityAvatar, an efficient framework for high-fidelity digital human modeling, specifically targeting loosely dressed avatars.<n>By incorporating a motion trend module and a latentbone encoder, we explicitly model pose-dependent deformations and temporal variations in clothing behavior.<n>Our method significantly enhances structural fidelity and perceptual quality in dynamic human reconstruction, particularly in non-rigid regions.
  arXiv  Detail & Related papers  (2025-04-02T09:59:12Z)
• FRESA: Feedforward Reconstruction of Personalized Skinned Avatars from Few Images [74.86864398919467]
  We present a novel method for reconstructing personalized 3D human avatars with realistic animation from only a few images.<n>We learn a universal prior from over a thousand clothed humans to achieve instant feedforward generation and zero-shot generalization.<n>Our method generates more authentic reconstruction and animation than state-of-the-arts, and can be directly generalized to inputs from casually taken phone photos.
  arXiv  Detail & Related papers  (2025-03-24T23:20:47Z)
• Surfel-based Gaussian Inverse Rendering for Fast and Relightable Dynamic Human Reconstruction from Monocular Video [54.88306920896417]
  SGIA introduces efficient training and rendering for relightable dynamic human reconstruction.<n>Our approach integrates pre-integration and image-based lighting for fast light calculations.<n>Experiments demonstrate that SGIA achieves highly accurate physical properties and significantly enhances the realistic relighting of dynamic human avatars.
  arXiv  Detail & Related papers  (2024-07-21T16:34:03Z)
• Relightable and Animatable Neural Avatars from Videos [14.091229306680697]
  We propose a method to create relightable and animatable neural avatars. The key challenge is to disentangle the geometry, material of the clothed body, and lighting. Experiments on synthetic and real datasets show that our approach reconstructs high-quality geometry.
  arXiv  Detail & Related papers  (2023-12-20T09:39:55Z)
• Relightable Neural Actor with Intrinsic Decomposition and Pose Control [80.06094206522668]
  We propose Relightable Neural Actor, a new video-based method for learning a pose-driven neural human model that can be relighted. For training, our method solely requires a multi-view recording of the human under a known, but static lighting condition. To evaluate our approach in real-world scenarios, we collect a new dataset with four identities recorded under different light conditions, indoors and outdoors.
  arXiv  Detail & Related papers  (2023-12-18T14:30:13Z)
• Relightable Gaussian Codec Avatars [26.255161061306428]
  We present Relightable Gaussian Codec Avatars, a method to build high-fidelity relightable head avatars that can be animated to generate novel expressions. Our geometry model based on 3D Gaussians can capture 3D-consistent sub-millimeter details such as hair strands and pores on dynamic face sequences. We improve the fidelity of eye reflections and enable explicit gaze control by introducing relightable explicit eye models.
  arXiv  Detail & Related papers  (2023-12-06T18:59:58Z)
• Towards Practical Capture of High-Fidelity Relightable Avatars [60.25823986199208]
  TRAvatar is trained with dynamic image sequences captured in a Light Stage under varying lighting conditions. It can predict the appearance in real-time with a single forward pass, achieving high-quality relighting effects. Our framework achieves superior performance for photorealistic avatar animation and relighting.
  arXiv  Detail & Related papers  (2023-09-08T10:26:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.