LumiMotion: Improving Gaussian Relighting with Scene Dynamics

• URL: http://arxiv.org/abs/2604.10994v1
• Date: Mon, 13 Apr 2026 04:50:05 GMT
• Title: LumiMotion: Improving Gaussian Relighting with Scene Dynamics
• Abstract summary: In 3D reconstruction, the problem of inverse rendering, namely recovering the illumination of the scene and the material properties, is fundamental.<n>Existing methods primarily target static scenes and assume simplified or moderate lighting to avoid entangling shadows with surface appearance.<n>We address this limitation by leveraging dynamic elements - regions of the scene that undergo motion - as a supervisory signal for inverse rendering.
• Score: 3.3825178252014525
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In 3D reconstruction, the problem of inverse rendering, namely recovering the illumination of the scene and the material properties, is fundamental. Existing Gaussian Splatting-based methods primarily target static scenes and often assume simplified or moderate lighting to avoid entangling shadows with surface appearance. This limits their ability to accurately separate lighting effects from material properties, particularly in real-world conditions. We address this limitation by leveraging dynamic elements - regions of the scene that undergo motion - as a supervisory signal for inverse rendering. Motion reveals the same surfaces under varying lighting conditions, providing stronger cues for disentangling material and illumination. This thesis is supported by our experimental results which show we improve LPIPS by 23% for albedo estimation and by 15% for scene relighting relative to next-best baseline. To this end, we introduce LumiMotion, the first Gaussian-based approach that leverages dynamics for inverse rendering and operates in arbitrary dynamic scenes. Our method learns a dynamic 2D Gaussian Splatting representation that employs a set of novel constraints which encourage the dynamic regions of the scene to deform, while keeping static regions stable. As we demonstrate, this separation is crucial for correct optimization of the albedo. Finally, we release a new synthetic benchmark comprising five scenes under four lighting conditions, each in both static and dynamic variants, for the first time enabling systematic evaluation of inverse rendering methods in dynamic environments and challenging lighting. Link to project page: https://joaxkal.github.io/LumiMotion/

Related papers

• SpotlessGS: Relightable 3D Gaussian Splatting under Dynamic Illumination for Robotic Perception [20.515607443256073]
  We extend Dark Gaussian Splatting to a more accurate and flexible relightable 3D reconstruction framework.<n>We introduce a low-frequency illumination model based on spherical harmonics to capture spatially varying residual and ambient lighting effects.<n> Experiments on synthetic and real-world datasets demonstrate that our method effectively mitigates illumination artifacts while improving rendering quality and quantitative performance over prior approaches.
  arXiv  Detail & Related papers  (2026-08-11T17:35:34Z)
• Flow4DGS-SLAM: Optical Flow-Guided 4D Gaussian Splatting SLAM [55.56111618153049]
  We propose an efficient framework for dynamic 3DGS SLAM guided by optical flow.<n>We boost the training speed of dynamic 3DGS by explicitly modeling their temporal centers.<n>The empirical results demonstrate our state-of-the-art performance in tracking, dynamic reconstruction, and training efficiency.
  arXiv  Detail & Related papers  (2026-04-24T08:13:01Z)
• LightHarmony3D: Harmonizing Illumination and Shadows for Object Insertion in 3D Gaussian Splatting [90.15229815175593]
  We present LightHarmony3D, a novel framework for illumination-consistent mesh insertion in 3DGS scenes.<n>By leveraging generative priors instead of iterative optimization, our method efficiently captures dominant scene illumination.<n>We introduce the first dedicated benchmark for mesh insertion in 3DGS, providing a standardized evaluation framework for assessing lighting consistency and photorealism.
  arXiv  Detail & Related papers  (2026-03-31T03:26:30Z)
• R3GW: Relightable 3D Gaussians for Outdoor Scenes in the Wild [23.68389428693905]
  3D Gaussian Splatting (3DGS) has established itself as a leading technique for 3D reconstruction and novel view synthesis of static scenes.<n>We present R3GW, a novel method that learns a relightable 3DGS representation of an outdoor scene captured in the wild.
  arXiv  Detail & Related papers  (2026-03-03T09:40:16Z)
• Divide-and-Conquer: Dual-Hierarchical Optimization for Semantic 4D Gaussian Spatting [16.15871890842964]
  We propose Dual-Hierarchical Optimization (DHO), which consists of Hierarchical Gaussian Flow and Hierarchical Gaussian Guidance.<n>Our method consistently outperforms the baselines on both synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2025-03-25T03:46:13Z)
• SpectroMotion: Dynamic 3D Reconstruction of Specular Scenes [7.590932716513324]
  We present a novel approach that combines 3D Gaussian Splatting (3DGS) with physically-based rendering (PBR) and deformation fields to reconstruct dynamic specular scenes.<n>It is the only existing 3DGS method capable of synthesizing photorealistic real-world dynamic specular scenes.
  arXiv  Detail & Related papers  (2024-10-22T17:59:56Z)
• NieR: Normal-Based Lighting Scene Rendering [17.421326290704844]
  NieR (Normal-Based Lighting Scene Rendering) is a novel framework that takes into account the nuances of light reflection on diverse material surfaces. We present the LD (Light Decomposition) module, which captures the lighting reflection characteristics on surfaces. We also propose the HNGD (Hierarchical Normal Gradient Densification) module to overcome the limitations of sparse Gaussian representation.
  arXiv  Detail & Related papers  (2024-05-21T14:24:43Z)
• GS-IR: 3D Gaussian Splatting for Inverse Rendering [71.14234327414086]
  We propose GS-IR, a novel inverse rendering approach based on 3D Gaussian Splatting (GS) We extend GS, a top-performance representation for novel view synthesis, to estimate scene geometry, surface material, and environment illumination from multi-view images captured under unknown lighting conditions. The flexible and expressive GS representation allows us to achieve fast and compact geometry reconstruction, photorealistic novel view synthesis, and effective physically-based rendering.
  arXiv  Detail & Related papers  (2023-11-26T02:35:09Z)
• BLiRF: Bandlimited Radiance Fields for Dynamic Scene Modeling [43.246536947828844]
  We propose a framework that factorizes time and space by formulating a scene as a composition of bandlimited, high-dimensional signals. We demonstrate compelling results across complex dynamic scenes that involve changes in lighting, texture and long-range dynamics.
  arXiv  Detail & Related papers  (2023-02-27T06:40:32Z)
• Neural Deformable Voxel Grid for Fast Optimization of Dynamic View Synthesis [63.25919018001152]
  We propose a fast deformable radiance field method to handle dynamic scenes. Our method achieves comparable performance to D-NeRF using only 20 minutes for training.
  arXiv  Detail & Related papers  (2022-06-15T17:49:08Z)
• Physically-Based Editing of Indoor Scene Lighting from a Single Image [106.60252793395104]
  We present a method to edit complex indoor lighting from a single image with its predicted depth and light source segmentation masks. We tackle this problem using two novel components: 1) a holistic scene reconstruction method that estimates scene reflectance and parametric 3D lighting, and 2) a neural rendering framework that re-renders the scene from our predictions.
  arXiv  Detail & Related papers  (2022-05-19T06:44:37Z)
• Learning Indoor Inverse Rendering with 3D Spatially-Varying Lighting [149.1673041605155]
  We address the problem of jointly estimating albedo, normals, depth and 3D spatially-varying lighting from a single image. Most existing methods formulate the task as image-to-image translation, ignoring the 3D properties of the scene. We propose a unified, learning-based inverse framework that formulates 3D spatially-varying lighting.
  arXiv  Detail & Related papers  (2021-09-13T15:29:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.