Joint Shadow Generation and Relighting via Light-Geometry Interaction Maps

• URL: http://arxiv.org/abs/2602.21820v2
• Date: Mon, 02 Mar 2026 01:20:14 GMT
• Title: Joint Shadow Generation and Relighting via Light-Geometry Interaction Maps
• Authors: Shan Wang, Peixia Li, Chenchen Xu, Ziang Cheng, Jiayu Yang, Hongdong Li, Pulak Purkait,
• Abstract summary: We propose Light-Geometry Interaction maps, a novel representation that encodes light-aware occlusion from monocular depth.<n>LGI captures essential light-shadow interactions reliably and accurately, computed from off-the-shelf 2.5D depth map predictions.<n>By embedding LGI into a bridge-matching generative backbone, we reduce ambiguity and enforce physically consistent light-shadow reasoning.
• Score: 51.82696819319878
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We propose Light-Geometry Interaction (LGI) maps, a novel representation that encodes light-aware occlusion from monocular depth. Unlike ray tracing, which requires full 3D reconstruction, LGI captures essential light-shadow interactions reliably and accurately, computed from off-the-shelf 2.5D depth map predictions. LGI explicitly ties illumination direction to geometry, providing a physics-inspired prior that constrains generative models. Without such prior, these models often produce floating shadows, inconsistent illumination, and implausible shadow geometry. Building on this representation, we propose a unified pipeline for joint shadow generation and relighting - unlike prior methods that treat them as disjoint tasks - capturing the intrinsic coupling of illumination and shadowing essential for modeling indirect effects. By embedding LGI into a bridge-matching generative backbone, we reduce ambiguity and enforce physically consistent light-shadow reasoning. To enable effective training, we curated the first large-scale benchmark dataset for joint shadow and relighting, covering reflections, transparency, and complex interreflections. Experiments show significant gains in realism and consistency across synthetic and real images. LGI thus bridges geometry-inspired rendering with generative modeling, enabling efficient, physically consistent shadow generation and relighting.

Related papers

• Physics-Grounded Attached Shadow Detection Using Approximate 3D Geometry and Light Direction [46.19532675330894]
  Attached shadows occur on the surface of objects where light cannot reach because of self-occlusion.<n>We introduce a framework that jointly detects cast and attached shadows by reasoning about their mutual relationship with scene illumination and geometry.<n>Our system consists of a shadow detection module that predicts both shadow types separately, and a light estimation module that infers the light direction from the detected shadows.
  arXiv  Detail & Related papers  (2025-12-05T22:01:27Z)
• Physics-Grounded Shadow Generation from Monocular 3D Geometry Priors and Approximate Light Direction [48.727438709248]
  In the physics of shadow formation, the occluder blocks some light rays casting from the light source that would otherwise arrive at the surface, creating a shadow that follows the silhouette of the occluder.<n>We propose a novel framework that embeds explicit physical modeling - geometry and illumination - into deep-learning-based shadow generation.
  arXiv  Detail & Related papers  (2025-12-05T21:52:23Z)
• KPLM-STA: Physically-Accurate Shadow Synthesis for Human Relighting via Keypoint-Based Light Modeling [10.273365471847102]
  We propose a novel shadow generation framework based on a Keypoints Linear Model (KPLM) and a Shadow Triangle Algorithm (STA)<n>KPLM models articulated human bodies using nine keypoints and one bounding block, enabling physically plausible shadow projection and dynamic shading across joints.<n>STA further improves geometric accuracy by computing shadow angles, lengths, and spatial positions through explicit geometric formulations.
  arXiv  Detail & Related papers  (2025-11-11T12:28:42Z)
• Generalizable and Relightable Gaussian Splatting for Human Novel View Synthesis [49.67420486373202]
  GRGS is a generalizable and relightable 3D Gaussian framework for high-fidelity human novel view synthesis under diverse lighting conditions.<n>We introduce a Lighting-aware Geometry Refinement (LGR) module trained on synthetically relit data to predict accurate depth and surface normals.
  arXiv  Detail & Related papers  (2025-05-27T17:59:47Z)
• GUS-IR: Gaussian Splatting with Unified Shading for Inverse Rendering [83.69136534797686]
  We present GUS-IR, a novel framework designed to address the inverse rendering problem for complicated scenes featuring rough and glossy surfaces. This paper starts by analyzing and comparing two prominent shading techniques popularly used for inverse rendering, forward shading and deferred shading. We propose a unified shading solution that combines the advantages of both techniques for better decomposition.
  arXiv  Detail & Related papers  (2024-11-12T01:51:05Z)
• GI-GS: Global Illumination Decomposition on Gaussian Splatting for Inverse Rendering [6.820642721852439]
  We present GI-GS, a novel inverse rendering framework that leverages 3D Gaussian Splatting (3DGS) and deferred shading.<n>In our framework, we first render a G-buffer to capture the detailed geometry and material properties of the scene.<n>With the G-buffer and previous rendering results, the indirect lighting can be calculated through a lightweight path tracing.
  arXiv  Detail & Related papers  (2024-10-03T15:58:18Z)
• Controllable Shadow Generation Using Pixel Height Maps [58.59256060452418]
  Physics-based shadow rendering methods require 3D geometries, which are not always available. Deep learning-based shadow synthesis methods learn a mapping from the light information to an object's shadow without explicitly modeling the shadow geometry. We introduce pixel heigh, a novel geometry representation that encodes the correlations between objects, ground, and camera pose.
  arXiv  Detail & Related papers  (2022-07-12T08:29:51Z)
• Geometry-aware Single-image Full-body Human Relighting [37.381122678376805]
  Single-image human relighting aims to relight a target human under new lighting conditions by decomposing the input image into albedo, shape and lighting. Previous methods suffer from both the entanglement between albedo and lighting and the lack of hard shadows. Our framework is able to generate photo-realistic high-frequency shadows such as cast shadows under challenging lighting conditions.
  arXiv  Detail & Related papers  (2022-07-11T10:21:02Z)
• DIB-R++: Learning to Predict Lighting and Material with a Hybrid Differentiable Renderer [78.91753256634453]
  We consider the challenging problem of predicting intrinsic object properties from a single image by exploiting differentiables. In this work, we propose DIBR++, a hybrid differentiable which supports these effects by combining specularization and ray-tracing. Compared to more advanced physics-based differentiables, DIBR++ is highly performant due to its compact and expressive model.
  arXiv  Detail & Related papers  (2021-10-30T01:59:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.