Shedding Light: A Benchmark for Evaluating Lighting Understanding in Generative Image Models

• URL: http://arxiv.org/abs/2609.10787v1
• Date: Wed, 09 Sep 2026 19:49:30 GMT
• Title: Shedding Light: A Benchmark for Evaluating Lighting Understanding in Generative Image Models
• Abstract summary: This work proposes a benchmark to assess the lighting understanding and harmonisation capabilities of generative models.<n>We use a multi-illumination dataset with images containing simple objects serving as light probes'', and prompt models to inpaint the same object onto the original image.<n>We then estimate the lighting direction, colour and radiance distribution from the inpainted probes, providing a quantitative measure of illumination accuracy and photometric realism.
• Score: 30.845505604996145
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate modelling of illumination is central to realistic image synthesis and scene understanding. Yet, there is little exploration into whether image generative models are good at this task or whether physical plausibility remains a key challenge for them. Clearly, significant progress has been made in realistic image synthesis, but do models truly understand lighting in a physically accurate manner? To answer this question, this work proposes a benchmark to assess the lighting understanding and harmonisation capabilities of generative models. Our key insight is that evaluating lighting understanding for such models only requires testing how well they insert novel objects into real photographs whilst maintaining consistent illumination. To do so, we use a multi-illumination dataset with images containing simple objects serving as ``light probes'', and prompt models to inpaint the same object onto the original image, then compare the generated results against the ground-truth light probes. We then estimate the lighting direction, colour and radiance distribution from the inpainted probes, providing a quantitative measure of illumination accuracy and photometric realism. Our work establishes a scalable evaluation protocol to systematically assess how well generative models capture and reproduce real-world lighting, offering a foundation for benchmarking the photometric accuracy of any future models. All code and data are available at https://lvsn.github.io/SheddingLight/ .

Related papers

• Do Image Editing Models Understand Lighting? [52.657799986035464]
  We introduce the 3D-anchored Light Probe (3DLP) benchmark, for which we have captured a new high-fidelity HDR dataset of real-world lighting changes.<n>The dataset consists of 1K image pairs of diverse indoor scenery in which light probes are physically turned on and off.<n>We evaluate a range of state-of-the-art image editing models by measuring how well their light probe edits align with reality.
  arXiv  Detail & Related papers  (2026-06-25T08:20:34Z)
• TokenLight: Precise Lighting Control in Images using Attribute Tokens [36.19968483255676]
  We formulate relighting as a conditional image generation task and introduce attribute tokens to encode lighting factors.<n>The model is trained on a large-scale synthetic dataset with ground-truth lighting annotations.<n>We validate our approach across a variety of relighting tasks, including controlling in-scene lighting fixtures and editing environment illumination.
  arXiv  Detail & Related papers  (2026-04-16T17:59:50Z)
• LightLab: Controlling Light Sources in Images with Diffusion Models [49.83835236202516]
  We present a diffusion-based method for fine-grained, parametric control over light sources in an image.<n>We leverage the linearity of light to synthesize image pairs depicting controlled light changes of either a target light source or ambient illumination.<n>We show how our method can achieve compelling light editing results, and outperforms existing methods based on user preference.
  arXiv  Detail & Related papers  (2025-05-14T17:57:27Z)
• Neural Gaffer: Relighting Any Object via Diffusion [43.87941408722868]
  We propose a novel end-to-end 2D relighting diffusion model, called Neural Gaffer. Our model takes a single image of any object and can synthesize an accurate, high-quality relit image under any novel lighting condition. We evaluate our model on both synthetic and in-the-wild Internet imagery and demonstrate its advantages in terms of generalization and accuracy.
  arXiv  Detail & Related papers  (2024-06-11T17:50:15Z)
• RelightableHands: Efficient Neural Relighting of Articulated Hand Models [46.60594572471557]
  We present the first neural relighting approach for rendering high-fidelity personalized hands that can be animated in real-time under novel illumination. Our approach adopts a teacher-student framework, where the teacher learns appearance under a single point light from images captured in a light-stage. Using images rendered by the teacher model as training data, an efficient student model directly predicts appearance under natural illuminations in real-time.
  arXiv  Detail & Related papers  (2023-02-09T18:59:48Z)
• Learning to Relight Portrait Images via a Virtual Light Stage and Synthetic-to-Real Adaptation [76.96499178502759]
  Relighting aims to re-illuminate the person in the image as if the person appeared in an environment with the target lighting. Recent methods rely on deep learning to achieve high-quality results. We propose a new approach that can perform on par with the state-of-the-art (SOTA) relighting methods without requiring a light stage.
  arXiv  Detail & Related papers  (2022-09-21T17:15:58Z)
• Self-supervised Outdoor Scene Relighting [92.20785788740407]
  We propose a self-supervised approach for relighting. Our approach is trained only on corpora of images collected from the internet without any user-supervision. Results show the ability of our technique to produce photo-realistic and physically plausible results, that generalizes to unseen scenes.
  arXiv  Detail & Related papers  (2021-07-07T09:46:19Z)
• Sparse Needlets for Lighting Estimation with Spherical Transport Loss [89.52531416604774]
  NeedleLight is a new lighting estimation model that represents illumination with needlets and allows lighting estimation in both frequency domain and spatial domain jointly. Extensive experiments show that NeedleLight achieves superior lighting estimation consistently across multiple evaluation metrics as compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-06-24T15:19:42Z)
• GMLight: Lighting Estimation via Geometric Distribution Approximation [86.95367898017358]
  This paper presents a lighting estimation framework that employs a regression network and a generative projector for effective illumination estimation. We parameterize illumination scenes in terms of the geometric light distribution, light intensity, ambient term, and auxiliary depth, and estimate them as a pure regression task. With the estimated lighting parameters, the generative projector synthesizes panoramic illumination maps with realistic appearance and frequency.
  arXiv  Detail & Related papers  (2021-02-20T03:31:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.