Do Image Editing Models Understand Lighting?

• URL: http://arxiv.org/abs/2606.26738v1
• Date: Thu, 25 Jun 2026 08:20:34 GMT
• Title: Do Image Editing Models Understand Lighting?
• Abstract summary: We introduce the 3D-anchored Light Probe (3DLP) benchmark, for which we have captured a new high-fidelity HDR dataset of real-world lighting changes.<n>The dataset consists of 1K image pairs of diverse indoor scenery in which light probes are physically turned on and off.<n>We evaluate a range of state-of-the-art image editing models by measuring how well their light probe edits align with reality.
• Score: 52.657799986035464
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While recent advancements in generative image editing models have achieved stunning visual fidelity, it remains an open question whether these systems possess an intrinsic knowledge of real-world lighting. Existing benchmarks typically evaluate high-level plausibility of perceptual light transport on curated internet imagery, using VLMs or human judgement, or they rely on synthetically generated datasets. In this work, we introduce the 3D-anchored Light Probe (3DLP) benchmark, for which we have captured a new high-fidelity HDR dataset of real-world lighting changes. The dataset consists of 1K image pairs of diverse indoor scenery in which light probes are physically turned on and off. To allow for a granular performance analysis, we annotated specific image regions such as cast shadows or metallic surfaces. With this data, we evaluate a range of state-of-the-art image editing models by measuring how well their light probe edits align with reality. The evaluation uses two new scores to compensate for AI-generated photographic effects, such as adjusted white balance. Our results show that the overall performance of models differs considerably, with differences slightly less pronounced for specular highlights. The best image editing models are remarkably consistent with real-world physics, however, they still leave room for improvement. We observe that image regions that receive less light from the light probe are more prone to errors for all models. Furthermore, building on their success in evaluating macroscopic lighting plausibility, we test VLMs on our task but find that they are unsuitable for pixel-level light transport analysis. We will make the benchmark, together with the real-world dataset, publicly available to encourage future research on this topic.

Related papers

• Shedding Light: A Benchmark for Evaluating Lighting Understanding in Generative Image Models [30.845505604996145]
  This work proposes a benchmark to assess the lighting understanding and harmonisation capabilities of generative models.<n>We use a multi-illumination dataset with images containing simple objects serving as light probes'', and prompt models to inpaint the same object onto the original image.<n>We then estimate the lighting direction, colour and radiance distribution from the inpainted probes, providing a quantitative measure of illumination accuracy and photometric realism.
  arXiv  Detail & Related papers  (2026-09-09T19:49:30Z)
• 3DPR: Single Image 3D Portrait Relight using Generative Priors [101.74130664920868]
  3DPR is an image-based relighting model that leverages generative priors learnt from multi-view One-Light-at-A-Time (OLAT) images.<n>We leverage the latent space of a pre-trained generative head model that provides a rich prior over face geometry learnt from in-the-wild image datasets.<n>Our reflectance network operates in the latent space of the generative head model, crucially enabling a relatively small number of lightstage images to train the reflectance model.
  arXiv  Detail & Related papers  (2025-10-17T17:37:42Z)
• LuxDiT: Lighting Estimation with Video Diffusion Transformer [66.60450792095901]
  Estimating scene lighting from a single image or video remains a longstanding challenge in computer vision and graphics.<n>We propose LuxDiT, a novel data-driven approach that fine-tunes a video diffusion transformer to generate HDR environment maps conditioned on visual input.
  arXiv  Detail & Related papers  (2025-09-03T19:59:20Z)
• Lite2Relight: 3D-aware Single Image Portrait Relighting [87.62069509622226]
  Lite2Relight is a novel technique that can predict 3D consistent head poses of portraits. By utilizing a pre-trained geometry-aware encoder and a feature alignment module, we map input images into a relightable 3D space. This includes producing 3D-consistent results of the full head, including hair, eyes, and expressions.
  arXiv  Detail & Related papers  (2024-07-15T07:16:11Z)
• Human Pose Estimation in Extremely Low-Light Conditions [21.210706205233286]
  We develop a dedicated camera system and build a new dataset of real low-light images with accurate pose labels. Thanks to our camera system, each low-light image in our dataset is coupled with an aligned well-lit image, which enables accurate pose labeling. We also propose a new model and a new training strategy that fully exploit the privileged information to learn representation insensitive to lighting conditions.
  arXiv  Detail & Related papers  (2023-03-27T17:28:25Z)
• DIB-R++: Learning to Predict Lighting and Material with a Hybrid Differentiable Renderer [78.91753256634453]
  We consider the challenging problem of predicting intrinsic object properties from a single image by exploiting differentiables. In this work, we propose DIBR++, a hybrid differentiable which supports these effects by combining specularization and ray-tracing. Compared to more advanced physics-based differentiables, DIBR++ is highly performant due to its compact and expressive model.
  arXiv  Detail & Related papers  (2021-10-30T01:59:39Z)
• NOD: Taking a Closer Look at Detection under Extreme Low-Light Conditions with Night Object Detection Dataset [25.29013780731876]
  Low light proves more difficult for machine cognition than previously thought. We present a large-scale dataset showing dynamic scenes captured on the streets at night. We propose to incorporate an image enhancement module into the object detection framework and two novel data augmentation techniques.
  arXiv  Detail & Related papers  (2021-10-20T03:44:04Z)
• Deep Portrait Lighting Enhancement with 3D Guidance [24.01582513386902]
  We present a novel deep learning framework for portrait lighting enhancement based on 3D facial guidance. Experimental results on the FFHQ dataset and in-the-wild images show that the proposed method outperforms state-of-the-art methods in terms of both quantitative metrics and visual quality.
  arXiv  Detail & Related papers  (2021-08-04T15:49:09Z)
• Neural Reflectance Fields for Appearance Acquisition [61.542001266380375]
  We present Neural Reflectance Fields, a novel deep scene representation that encodes volume density, normal and reflectance properties at any 3D point in a scene. We combine this representation with a physically-based differentiable ray marching framework that can render images from a neural reflectance field under any viewpoint and light.
  arXiv  Detail & Related papers  (2020-08-09T22:04:36Z)
• A study of the effect of the illumination model on the generation of synthetic training datasets [4.238220229373858]
  We study how the illumination model used by the rendering software affects the quality of the generated images. We created eight training sets, each one with a different illumination model, and tested them on three different network architectures.
  arXiv  Detail & Related papers  (2020-06-15T23:22:24Z)
• Deep Lighting Environment Map Estimation from Spherical Panoramas [0.0]
  We present a data-driven model that estimates an HDR lighting environment map from a single LDR monocular spherical panorama. We exploit the availability of surface geometry to employ image-based relighting as a data generator and supervision mechanism.
  arXiv  Detail & Related papers  (2020-05-16T14:23:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.