Related papers: Spatiotemporally Consistent Indoor Lighting Estimation with Diffusion Priors

Spatiotemporally Consistent Indoor Lighting Estimation with Diffusion Priors

URL: http://arxiv.org/abs/2508.08384v1
Date: Mon, 11 Aug 2025 18:11:42 GMT
Title: Spatiotemporally Consistent Indoor Lighting Estimation with Diffusion Priors
Authors: Mutian Tong, Rundi Wu, Changxi Zheng,
Abstract summary: Lighting estimation from a single image or video remains a challenge due to its highly ill-posed nature.<n>We propose a method that estimates from an input video describing a continuous light field describing lighting of the scene.<n>Results on consistent lighting estimation from in-the-wild videos, which is rarely demonstrated in previous works.
Score: 18.794530505630227
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: Indoor lighting estimation from a single image or video remains a challenge due to its highly ill-posed nature, especially when the lighting condition of the scene varies spatially and temporally. We propose a method that estimates from an input video a continuous light field describing the spatiotemporally varying lighting of the scene. We leverage 2D diffusion priors for optimizing such light field represented as a MLP. To enable zero-shot generalization to in-the-wild scenes, we fine-tune a pre-trained image diffusion model to predict lighting at multiple locations by jointly inpainting multiple chrome balls as light probes. We evaluate our method on indoor lighting estimation from a single image or video and show superior performance over compared baselines. Most importantly, we highlight results on spatiotemporally consistent lighting estimation from in-the-wild videos, which is rarely demonstrated in previous works.

Related papers

Lighting in Motion: Spatiotemporal HDR Lighting Estimation [17.395631978283657]
We present Lighting in Motion (LiMo), a diffusion-based approach to lighting estimation.<n>LiMo is both realistic high-frequency prediction and accurate illuminance estimation.
arXiv Detail & Related papers (2025-12-15T17:49:22Z)
LightLab: Controlling Light Sources in Images with Diffusion Models [49.83835236202516]
We present a diffusion-based method for fine-grained, parametric control over light sources in an image.<n>We leverage the linearity of light to synthesize image pairs depicting controlled light changes of either a target light source or ambient illumination.<n>We show how our method can achieve compelling light editing results, and outperforms existing methods based on user preference.
arXiv Detail & Related papers (2025-05-14T17:57:27Z)
Neural LightRig: Unlocking Accurate Object Normal and Material Estimation with Multi-Light Diffusion [45.81230812844384]
We present a novel framework that boosts intrinsic estimation by leveraging auxiliary multi-lighting conditions from 2D diffusion priors.<n>We train a large G-buffer model with a U-Net backbone to accurately predict surface normals and materials.
arXiv Detail & Related papers (2024-12-12T18:58:09Z)
Spatiotemporally Consistent HDR Indoor Lighting Estimation [66.26786775252592]
We propose a physically-motivated deep learning framework to solve the indoor lighting estimation problem. Given a single LDR image with a depth map, our method predicts spatially consistent lighting at any given image position. Our framework achieves photorealistic lighting prediction with higher quality compared to state-of-the-art single-image or video-based methods.
arXiv Detail & Related papers (2023-05-07T20:36:29Z)
Progressively Optimized Local Radiance Fields for Robust View Synthesis [76.55036080270347]
We present an algorithm for reconstructing the radiance field of a large-scale scene from a single casually captured video. For handling unknown poses, we jointly estimate the camera poses with radiance field in a progressive manner. For handling large unbounded scenes, we dynamically allocate new local radiance fields trained with frames within a temporal window.
arXiv Detail & Related papers (2023-03-24T04:03:55Z)
Neural Light Field Estimation for Street Scenes with Differentiable Virtual Object Insertion [129.52943959497665]
Existing works on outdoor lighting estimation typically simplify the scene lighting into an environment map. We propose a neural approach that estimates the 5D HDR light field from a single image. We show the benefits of our AR object insertion in an autonomous driving application.
arXiv Detail & Related papers (2022-08-19T17:59:16Z)
Spatio-Temporal Outdoor Lighting Aggregation on Image Sequences using Transformer Networks [23.6427456783115]
In this work, we focus on outdoor lighting estimation by aggregating individual noisy estimates from images. Recent work based on deep neural networks has shown promising results for single image lighting estimation, but suffers from robustness. We tackle this problem by combining lighting estimates from several image views sampled in the angular and temporal domain of an image sequence.
arXiv Detail & Related papers (2022-02-18T14:11:16Z)
Neural Radiance Fields for Outdoor Scene Relighting [70.97747511934705]
We present NeRF-OSR, the first approach for outdoor scene relighting based on neural radiance fields. In contrast to the prior art, our technique allows simultaneous editing of both scene illumination and camera viewpoint. It also includes a dedicated network for shadow reproduction, which is crucial for high-quality outdoor scene relighting.
arXiv Detail & Related papers (2021-12-09T18:59:56Z)
Sparse Needlets for Lighting Estimation with Spherical Transport Loss [89.52531416604774]
NeedleLight is a new lighting estimation model that represents illumination with needlets and allows lighting estimation in both frequency domain and spatial domain jointly. Extensive experiments show that NeedleLight achieves superior lighting estimation consistently across multiple evaluation metrics as compared with state-of-the-art methods.
arXiv Detail & Related papers (2021-06-24T15:19:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.