Light-X: Generative 4D Video Rendering with Camera and Illumination Control

• URL: http://arxiv.org/abs/2512.05115v1
• Date: Thu, 04 Dec 2025 18:59:57 GMT
• Title: Light-X: Generative 4D Video Rendering with Camera and Illumination Control
• Authors: Tianqi Liu, Zhaoxi Chen, Zihao Huang, Shaocong Xu, Saining Zhang, Chongjie Ye, Bohan Li, Zhiguo Cao, Wei Li, Hao Zhao, Ziwei Liu,
• Abstract summary: Light-X is a video generation framework that enables controllable rendering from monocular videos with both viewpoint and illumination control.<n>To address the lack of paired multi-view and multi-illumination videos, we introduce Light-Syn, a degradation-based pipeline with inverse-mapping.
• Score: 52.87059646145144
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in illumination control extend image-based methods to video, yet still facing a trade-off between lighting fidelity and temporal consistency. Moving beyond relighting, a key step toward generative modeling of real-world scenes is the joint control of camera trajectory and illumination, since visual dynamics are inherently shaped by both geometry and lighting. To this end, we present Light-X, a video generation framework that enables controllable rendering from monocular videos with both viewpoint and illumination control. 1) We propose a disentangled design that decouples geometry and lighting signals: geometry and motion are captured via dynamic point clouds projected along user-defined camera trajectories, while illumination cues are provided by a relit frame consistently projected into the same geometry. These explicit, fine-grained cues enable effective disentanglement and guide high-quality illumination. 2) To address the lack of paired multi-view and multi-illumination videos, we introduce Light-Syn, a degradation-based pipeline with inverse-mapping that synthesizes training pairs from in-the-wild monocular footage. This strategy yields a dataset covering static, dynamic, and AI-generated scenes, ensuring robust training. Extensive experiments show that Light-X outperforms baseline methods in joint camera-illumination control and surpasses prior video relighting methods under both text- and background-conditioned settings.

Related papers

• Joint Shadow Generation and Relighting via Light-Geometry Interaction Maps [51.82696819319878]
  We propose Light-Geometry Interaction maps, a novel representation that encodes light-aware occlusion from monocular depth.<n>LGI captures essential light-shadow interactions reliably and accurately, computed from off-the-shelf 2.5D depth map predictions.<n>By embedding LGI into a bridge-matching generative backbone, we reduce ambiguity and enforce physically consistent light-shadow reasoning.
  arXiv  Detail & Related papers  (2026-02-25T11:47:26Z)
• POLAR: A Portrait OLAT Dataset and Generative Framework for Illumination-Aware Face Modeling [51.7495375918484]
  Face relighting aims to synthesize realistic portraits under novel illumination while preserving identity and geometry.<n>We introduce POLAR, a large-scale and physically calibrated One-Light-at-a-Time dataset containing over 200 subjects captured under 156 lighting directions, multiple views, and diverse expressions.<n>We develop a flow-based generative model POLARNet that predicts per-light OLAT responses from a single portrait, capturing fine-grained and direction-aware illumination effects while preserving facial identity.
  arXiv  Detail & Related papers  (2025-12-15T11:04:09Z)
• GLOW: Global Illumination-Aware Inverse Rendering of Indoor Scenes Captured with Dynamic Co-Located Light & Camera [18.90141473604964]
  Inverse rendering of indoor scenes remains challenging due to the ambiguity between reflectance and lighting.<n>We present GLOW, a Global Illumination-aware Inverse Rendering framework designed to address these challenges.
  arXiv  Detail & Related papers  (2025-11-28T03:24:12Z)
• ReLumix: Extending Image Relighting to Video via Video Diffusion Models [5.890782804843724]
  Controlling illumination during video post-production is a crucial yet elusive goal in computational photography.<n>This paper introduces ReLumix, a novel framework that decouples the relighting from temporal synthesis.<n>Although trained on synthetic data, ReLumix shows competitive generalization to real-world videos.
  arXiv  Detail & Related papers  (2025-09-28T09:35:33Z)
• IllumiCraft: Unified Geometry and Illumination Diffusion for Controllable Video Generation [79.1960960864242]
  IllumiCraft is an end-to-end diffusion framework accepting three complementary inputs.<n>It generates temporally coherent videos aligned with user-defined prompts.
  arXiv  Detail & Related papers  (2025-06-03T17:59:52Z)
• Light-A-Video: Training-free Video Relighting via Progressive Light Fusion [52.420894727186216]
  Light-A-Video is a training-free approach to achieve temporally smooth video relighting.<n>Adapted from image relighting models, Light-A-Video introduces two key techniques to enhance lighting consistency.
  arXiv  Detail & Related papers  (2025-02-12T17:24:19Z)
• AKiRa: Augmentation Kit on Rays for optical video generation [9.255424148510572]
  AKiRa is a novel augmentation framework that builds and trains a camera adapter with a complex camera model over an existing video generation backbone.<n>It enables fine-tuned control over camera motion as well as complex optical parameters to achieve cinematic effects such as zoom, fisheye effect, and bokeh.<n>This work sets a new landmark in controlled and optically enhanced video generation, paving the way for future optical video generation methods.
  arXiv  Detail & Related papers  (2024-12-18T18:53:22Z)
• LumiSculpt: Enabling Consistent Portrait Lighting in Video Generation [87.95655555555264]
  Lighting plays a pivotal role in ensuring the naturalness and aesthetic quality of video generation.<n>LumiSculpt enables precise and consistent lighting control in T2V generation models.<n>LumiHuman is a new dataset for portrait lighting of images and videos.
  arXiv  Detail & Related papers  (2024-10-30T12:44:08Z)
• Neural Video Portrait Relighting in Real-time via Consistency Modeling [41.04622998356025]
  We propose a neural approach for real-time, high-quality and coherent video portrait relighting. We propose a hybrid structure and lighting disentanglement in an encoder-decoder architecture. We also propose a lighting sampling strategy to model the illumination consistency and mutation for natural portrait light manipulation in real-world.
  arXiv  Detail & Related papers  (2021-04-01T14:13:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.