SRUG: Shadow-Guided Relightable Urban Scene with Generation Model

• URL: http://arxiv.org/abs/2605.24700v3
• Date: Fri, 29 May 2026 01:32:54 GMT
• Title: SRUG: Shadow-Guided Relightable Urban Scene with Generation Model
• Abstract summary: Shadow-guided Relightable Urban Scene with Generation model (SRUG) is a novel framework designed to address relighting challenges in urban scenes.<n>SRUG leverages shadows to guide a 3D completion model for recovering the geometry of invisible regions.<n>We introduce a physically-based lighting model that captures the complex illumination of urban scenes and supports reliable relighting.
• Score: 26.493362508040978
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Creating relightable urban scenes from images or videos is widely useful but highly ill-posed. Urban environments are typically unbounded and extend beyond the visible regions. As a result, many portions of the scene remain unobserved, yet these invisible regions can cast shadows onto visible areas. Reasonably modeling shadows cast by such invisible regions is challenging and poses a significant obstacle to creating relightable urban scenes. At the same time, sparse input views and complex illumination conditions further complicate relighting, as they introduce severe ambiguities in material decomposition. In this paper, we propose Shadow-guided Relightable Urban Scene with Generation model (SRUG), a novel framework designed to address relighting challenges in urban scenes. SRUG leverages shadows to guide a 3D completion model for recovering the geometry of invisible regions, promoting the synthesis of physically reasonable shadows. In addition, SRUG employs an iterative material decomposition scheme that applies the large material model (LMM) to provide material supervision and iteratively decompose the scene's material properties, enabling robust material decomposition. Building upon these components, we introduce a physically-based lighting model that captures the complex illumination of urban scenes and supports reliable relighting. Extensive quantitative evaluations and visual comparisons demonstrate that our method outperforms existing approaches in both novel view synthesis and relighting tasks.

Related papers

• WildShadowRemover: In-the-Wild Video Shadow Removal via Detail-Preserving Video Diffusion Models [32.587071160357866]
  Video shadow removal in the wild remains challenging due to complex illumination, diverse shadow appearances, and limited training data.<n>We present WildShadowRemover, a framework that adapts a pretrained video diffusion model for robust video shadow removal via LoRA fine-tuning.<n>We also construct WildShadow, a large-scale paired video shadow removal dataset and benchmark, covering diverse synthetic scenes.
  arXiv  Detail & Related papers  (2026-07-28T19:08:26Z)
• IG-Diff: Complex Night Scene Restoration with Illumination-Guided Diffusion Model [113.62837263178346]
  In nighttime circumstances, it is challenging for individuals and machines to perceive their surroundings.<n>We propose an integration of an illumination-guided module embedded in the diffusion model to guide the illumination restoration process.<n>Our model can preserve texture fidelity while contending with the adversities posed by various degradation in low-light scenarios.
  arXiv  Detail & Related papers  (2026-05-14T04:01:41Z)
• LightHarmony3D: Harmonizing Illumination and Shadows for Object Insertion in 3D Gaussian Splatting [90.15229815175593]
  We present LightHarmony3D, a novel framework for illumination-consistent mesh insertion in 3DGS scenes.<n>By leveraging generative priors instead of iterative optimization, our method efficiently captures dominant scene illumination.<n>We introduce the first dedicated benchmark for mesh insertion in 3DGS, providing a standardized evaluation framework for assessing lighting consistency and photorealism.
  arXiv  Detail & Related papers  (2026-03-31T03:26:30Z)
• LightCity: An Urban Dataset for Outdoor Inverse Rendering and Reconstruction under Multi-illumination Conditions [80.70675855203154]
  Inverse rendering in urban scenes is pivotal for applications like autonomous driving and digital twins.<n>Yet, it faces significant challenges due to complex illumination conditions, including multi-illumination and indirect light and shadow effects.<n>We present LightCity, a novel high-quality synthetic urban dataset featuring diverse illumination conditions with realistic indirect light and shadow effects.
  arXiv  Detail & Related papers  (2026-02-01T09:37:00Z)
• MaterialRefGS: Reflective Gaussian Splatting with Multi-view Consistent Material Inference [83.38607296779423]
  We show that multi-view consistent material inference with more physically-based environment modeling is key to learning accurate reflections with Gaussian Splatting.<n>Our method faithfully recovers both illumination and geometry, achieving state-of-the-art rendering quality in novel views synthesis.
  arXiv  Detail & Related papers  (2025-10-13T13:29:20Z)
• Gaussian Shadow Casting for Neural Characters [20.78790953284832]
  We propose a new shadow model using a Gaussian density proxy that replaces sampling with a simple analytic formula. It supports dynamic motion and is tailored for shadow computation, thereby avoiding the affine projection approximation and sorting required by the closely related Gaussian splatting. We demonstrate improved reconstructions, with better separation of albedo, shading, and shadows in challenging outdoor scenes with direct sun light and hard shadows.
  arXiv  Detail & Related papers  (2024-01-11T18:50:31Z)
• GS-IR: 3D Gaussian Splatting for Inverse Rendering [71.14234327414086]
  We propose GS-IR, a novel inverse rendering approach based on 3D Gaussian Splatting (GS) We extend GS, a top-performance representation for novel view synthesis, to estimate scene geometry, surface material, and environment illumination from multi-view images captured under unknown lighting conditions. The flexible and expressive GS representation allows us to achieve fast and compact geometry reconstruction, photorealistic novel view synthesis, and effective physically-based rendering.
  arXiv  Detail & Related papers  (2023-11-26T02:35:09Z)
• UrbanIR: Large-Scale Urban Scene Inverse Rendering from a Single Video [22.613946218766802]
  We present UrbanIR, a new inverse graphics model that enables realistic, free-viewpoint renderings of scenes under various lighting conditions with a single video.<n>It accurately infers shape, albedo, visibility, and sun and sky illumination from wide-baseline videos, such as those from car-mounted cameras.<n>UrbanIR addresses these issues with novel losses that reduce errors in inverse graphics inference and rendering artifacts.
  arXiv  Detail & Related papers  (2023-06-15T17:59:59Z)
• Neural Fields meet Explicit Geometric Representation for Inverse Rendering of Urban Scenes [62.769186261245416]
  We present a novel inverse rendering framework for large urban scenes capable of jointly reconstructing the scene geometry, spatially-varying materials, and HDR lighting from a set of posed RGB images with optional depth. Specifically, we use a neural field to account for the primary rays, and use an explicit mesh (reconstructed from the underlying neural field) for modeling secondary rays that produce higher-order lighting effects such as cast shadows.
  arXiv  Detail & Related papers  (2023-04-06T17:51:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.