Video Generation Models Are Inherent Lighting Estimators

• URL: http://arxiv.org/abs/2607.04674v1
• Date: Mon, 06 Jul 2026 04:59:17 GMT
• Title: Video Generation Models Are Inherent Lighting Estimators
• Abstract summary: Recent generation models can produce scenes with complex lighting, possessing an inherent understanding of lighting.<n>We introduce V-LITE (Video generation models are inherent lighting estimators), a framework that unlocks this internal knowledge by reframing estimation.<n>To bridge the gap from LDR-native models to the HDR domain, we design an HDR-aware VAE and employ an efficient LoRA-based fine-tuning strategy.
• Score: 54.44164570819254
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recovering dynamic environment maps from a single in-the-wild video is crucial for photorealistic rendering, yet remains a challenge. Recent video generation models can produce photorealistic scenes with complex lighting, possessing an inherent understanding of lighting. In this paper, we introduce V-LITE (Video generation models are inherent lighting estimators), a framework that unlocks this internal knowledge by reframing lighting estimation as a guided video inpainting task. Inspired by VFX industry practices, we insert a synthetic chrome ball into the scene to compel the model to generate physically plausible reflections from the surrounding spatio-temporal context. To bridge the gap from LDR-native models to the HDR domain, we design an HDR-aware VAE and employ an efficient LoRA-based fine-tuning strategy. We then construct a mixed dataset comprising high-fidelity HDR images to provide realistic HDR priors, and in-the-wild HDR videos to provide dynamic spatio-temporal context. Extensive experiments demonstrate that V-LITE produces temporally coherent HDR environment maps, revealing that modern video diffusion models are not merely synthesizers but also powerful, inherently capable estimators of physical scene lighting.

Related papers

• HDR Video Generation via Latent Alignment with Logarithmic Encoding [41.63873187838369]
  We demonstrate high-quality HDR video generation using a pretrained video model with minimal adaptation.<n>Our results indicate that HDR, despite representing a fundamentally different image formation regime, can be handled effectively without redesigning generative models.
  arXiv  Detail & Related papers  (2026-04-13T17:55:02Z)
• DiffHDR: Re-Exposing LDR Videos with Video Diffusion Models [35.73610036182625]
  Diff is a framework that formulates LDR-to-generative conversion within the latent space of a video diffusion model.<n>Our framework further enables controllable LDR-to-video conversion guided by text or reference images.
  arXiv  Detail & Related papers  (2026-04-07T17:56:18Z)
• Physically Inspired Gaussian Splatting for HDR Novel View Synthesis [40.99906396910274]
  High dynamic range novel view synthesis (biased-NVS) reconstructs scenes with dynamic details by fusing multi-exposure low dynamic range (LDR) views.<n>Implicitly HDR content by constraining tone-mapped results fails in correcting abnormal HDR values, and results in limited gradients for Gaussians in under/over-exposed regions.<n>We introduce Phys-GS, a physically inspired HDR-NVS framework that models scene appearance via intrinsic reflectance and adjustable ambient illumination.
  arXiv  Detail & Related papers  (2026-03-30T04:27:39Z)
• Dynamic Novel View Synthesis in High Dynamic Range [78.72910306733607]
  Current methods primarily focus on static scenes, implicitly assuming all scene elements remain stationary and non-living.<n>We introduce HDR-4DGS, a Gaussian Splatting-based architecture featured with an innovative dynamic tone-mapping module.<n>Experiments demonstrate that HDR-4DGS surpasses existing state-of-the-art methods in both quantitative performance and visual fidelity.
  arXiv  Detail & Related papers  (2025-09-26T04:29:22Z)
• LuxDiT: Lighting Estimation with Video Diffusion Transformer [66.60450792095901]
  Estimating scene lighting from a single image or video remains a longstanding challenge in computer vision and graphics.<n>We propose LuxDiT, a novel data-driven approach that fine-tunes a video diffusion transformer to generate HDR environment maps conditioned on visual input.
  arXiv  Detail & Related papers  (2025-09-03T19:59:20Z)
• LEDiff: Latent Exposure Diffusion for HDR Generation [11.669442066168244]
  LEDiff is a method that enables a generative model with HDR content generation through latent space exposure fusion techniques.<n>It also functions as an LDR-to- fusion converter, expanding the dynamic range of existing low-dynamic range images.
  arXiv  Detail & Related papers  (2024-12-19T02:15:55Z)
• Cinematic Gaussians: Real-Time HDR Radiance Fields with Depth of Field [23.92087253022495]
  Radiance field methods represent the state of the art in reconstructing complex scenes from multi-view photos. Their reliance on a pinhole camera model, assuming all scene elements are in focus in the input images, presents practical challenges and complicates refocusing during novel-view synthesis. We present a lightweight analytical based on 3D Gaussian Splatting that utilizes multi-view LDR images on varying exposure times, radiance of apertures, and focus distances as input to reconstruct a high-dynamic-range scene.
  arXiv  Detail & Related papers  (2024-06-11T15:00:24Z)
• Spatiotemporally Consistent HDR Indoor Lighting Estimation [66.26786775252592]
  We propose a physically-motivated deep learning framework to solve the indoor lighting estimation problem. Given a single LDR image with a depth map, our method predicts spatially consistent lighting at any given image position. Our framework achieves photorealistic lighting prediction with higher quality compared to state-of-the-art single-image or video-based methods.
  arXiv  Detail & Related papers  (2023-05-07T20:36:29Z)
• GlowGAN: Unsupervised Learning of HDR Images from LDR Images in the Wild [74.52723408793648]
  We present the first method for learning a generative model of HDR images from in-the-wild LDR image collections in a fully unsupervised manner. The key idea is to train a generative adversarial network (GAN) to generate HDR images which, when projected to LDR under various exposures, are indistinguishable from real LDR images. Experiments show that our method GlowGAN can synthesize photorealistic HDR images in many challenging cases such as landscapes, lightning, or windows.
  arXiv  Detail & Related papers  (2022-11-22T15:42:08Z)
• StyleLight: HDR Panorama Generation for Lighting Estimation and Editing [98.20167223076756]
  We present a new lighting estimation and editing framework to generate high-dynamic-range (GAN) indoor panorama lighting from a single field-of-view (LFOV) image. Our framework achieves superior performance over state-of-the-art methods on indoor lighting estimation.
  arXiv  Detail & Related papers  (2022-07-29T17:58:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.