LightFuse: Relightable Interactive Gaussian Scene Reconstruction via Multi-Scan Fusion and 2D Gaussian Ray Tracing

• URL: http://arxiv.org/abs/2608.29269v1
• Date: Sat, 29 Aug 2026 13:49:13 GMT
• Title: LightFuse: Relightable Interactive Gaussian Scene Reconstruction via Multi-Scan Fusion and 2D Gaussian Ray Tracing
• Abstract summary: Relightable interactive scene reconstruction aims to build an editable 3D model from scans of different object arrangements and render new layouts under novel illumination.<n>We present LightFuse, a 2D Gaussian framework that extends interactive scene reconstruction with explicit material-illumination decomposition and physically based relighting.
• Score: 19.013915783052433
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Relightable interactive scene reconstruction aims to build an editable 3D model from scans of different object arrangements and render new layouts under novel illumination. Existing methods either bake lighting into appearance or recover material and illumination only for fixed scenes, leaving edited layouts with inconsistent shadows and indirect lighting. We present LightFuse, a 2D Gaussian framework that extends interactive scene reconstruction with explicit material-illumination decomposition and physically based relighting. LightFuse first fuses observations across states to reconstruct a shared background and movable objects. It then conducts ray-tracing-oriented geometry refinement to produce more complete and consistent surfaces. On the refined geometry, staged training with differentiable one-bounce ray tracing separates shared metallic--roughness material from state-specific environment lighting. The resulting scene supports object rearrangement, material editing, and relighting, while ray tracing recomputes appearance after each interaction. Experiments across synthetic scenes demonstrate state-of-the-art relighting quality, outperforming the strongest baseline by +9.74\,dB PSNR and +0.121 SSIM on average. Project page: https://zhn202.github.io/LightFuse/

Related papers

• Relightable Gaussian Splatting for Virtual Production Using Image-Based Illumination [5.673030658112764]
  Virtual production (VP) use LED walls to provide both background imagery and image-based lighting.<n>In inverse rendering, physically-based rendering relies on estimates 3D geometry and lighting, using environment maps.<n>We propose a VP-specific framework for 3D reconstruction and relighting using Gaussian Splatting.
  arXiv  Detail & Related papers  (2026-05-09T16:04:06Z)
• LightHarmony3D: Harmonizing Illumination and Shadows for Object Insertion in 3D Gaussian Splatting [90.15229815175593]
  We present LightHarmony3D, a novel framework for illumination-consistent mesh insertion in 3DGS scenes.<n>By leveraging generative priors instead of iterative optimization, our method efficiently captures dominant scene illumination.<n>We introduce the first dedicated benchmark for mesh insertion in 3DGS, providing a standardized evaluation framework for assessing lighting consistency and photorealism.
  arXiv  Detail & Related papers  (2026-03-31T03:26:30Z)
• Rotated Lights for Consistent and Efficient 2D Gaussians Inverse Rendering [14.904694518700254]
  Inverse rendering aims to decompose a scene into its geometry, material properties and light conditions under a certain rendering model.<n>In recent years, inverse rendering methods have been inspired by view synthesis approaches like neural radiance fields and Gaussian splatting.<n>We propose RotLight, a simple capturing setup, to address the ambiguity.
  arXiv  Detail & Related papers  (2026-02-09T14:34:06Z)
• EAG-PT: Emission-Aware Gaussians and Path Tracing for Indoor Scene Reconstruction and Editing [63.64931392651423]
  We propose Emission-Aware Gaussians and Path Tracing (EAG-PT), aiming for physically based light transport with a unified 2D Gaussian representation.<n>Our design is based on three cores: (1) using 2D Gaussians as a unified scene representation and transport-friendly geometry proxy that avoids reconstructed mesh, (2) explicitly separating emissive and non-emissive components during reconstruction for further scene editing, and (3) decoupling reconstruction from final rendering by using efficient single-bounce optimization and high-quality multi-bounce path tracing after scene editing.
  arXiv  Detail & Related papers  (2026-01-30T15:16:37Z)
• ComGS: Efficient 3D Object-Scene Composition via Surface Octahedral Probes [46.83857963152283]
  Gaussian Splatting (GS) enables immersive rendering, but realistic 3D object-scene composition remains challenging.<n>We propose ComGS, a novel 3D object-scene composition framework.<n>Our method achieves high-quality, real-time rendering at around 28 FPS, produces visually harmonious results with vivid shadows, and requires only 36 seconds for editing.
  arXiv  Detail & Related papers  (2025-10-09T03:10:41Z)
• DeferredGS: Decoupled and Editable Gaussian Splatting with Deferred Shading [50.331929164207324]
  We introduce DeferredGS, a method for decoupling and editing the Gaussian splatting representation using deferred shading. Both qualitative and quantitative experiments demonstrate the superior performance of DeferredGS in novel view and editing tasks.
  arXiv  Detail & Related papers  (2024-04-15T01:58:54Z)
• Neural Fields meet Explicit Geometric Representation for Inverse Rendering of Urban Scenes [62.769186261245416]
  We present a novel inverse rendering framework for large urban scenes capable of jointly reconstructing the scene geometry, spatially-varying materials, and HDR lighting from a set of posed RGB images with optional depth. Specifically, we use a neural field to account for the primary rays, and use an explicit mesh (reconstructed from the underlying neural field) for modeling secondary rays that produce higher-order lighting effects such as cast shadows.
  arXiv  Detail & Related papers  (2023-04-06T17:51:54Z)
• Physically-Based Editing of Indoor Scene Lighting from a Single Image [106.60252793395104]
  We present a method to edit complex indoor lighting from a single image with its predicted depth and light source segmentation masks. We tackle this problem using two novel components: 1) a holistic scene reconstruction method that estimates scene reflectance and parametric 3D lighting, and 2) a neural rendering framework that re-renders the scene from our predictions.
  arXiv  Detail & Related papers  (2022-05-19T06:44:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.