LGTM: Training-Free Light-Guided Text-to-Image Diffusion Model via Initial Noise Manipulation

• URL: http://arxiv.org/abs/2603.24086v1
• Date: Wed, 25 Mar 2026 08:46:31 GMT
• Title: LGTM: Training-Free Light-Guided Text-to-Image Diffusion Model via Initial Noise Manipulation
• Abstract summary: We propose a novel Training-Free Light-Guided Text-to-Image Diffusion Model via Initial Noise Manipulation.<n>It manipulates the initial latent noise of the diffusion process to guide image generation with text prompts and user-specified light directions.<n>It surpasses prompt-based baselines in lighting consistency, while preserving image quality and text alignment.
• Score: 7.432606687404551
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion models have demonstrated high-quality performance in conditional text-to-image generation, particularly with structural cues such as edges, layouts, and depth. However, lighting conditions have received limited attention and remain difficult to control within the generative process. Existing methods handle lighting through a two-stage pipeline that relights images after generation, which is inefficient. Moreover, they rely on fine-tuning with large datasets and heavy computation, limiting their adaptability to new models and tasks. To address this, we propose a novel Training-Free Light-Guided Text-to-Image Diffusion Model via Initial Noise Manipulation (LGTM), which manipulates the initial latent noise of the diffusion process to guide image generation with text prompts and user-specified light directions. Through a channel-wise analysis of the latent space, we find that selectively manipulating latent channels enables fine-grained lighting control without fine-tuning or modifying the pre-trained model. Extensive experiments show that our method surpasses prompt-based baselines in lighting consistency, while preserving image quality and text alignment. This approach introduces new possibilities for dynamic, user-guided light control. Furthermore, it integrates seamlessly with models like ControlNet, demonstrating adaptability across diverse scenarios.

Related papers

• Dior: Drawing the Light of Image via Material-Decoupled Illumination Representation [12.37976886129991]
  Controllable image relighting is an important problem in image editing, and hand-drawn scribbles provide an intuitive interface for specifying the desired illumination.<n>We introduce a material-decoupled illumination representation, termed the Lumi Map, which establishes an explicit mapping between user scribbles and the resulting illumination.<n>We then present Dior-Light, an image relighting method controlled by hand-drawn strokes.
  arXiv  Detail & Related papers  (2026-08-30T17:41:17Z)
• Diffusion-Based Low-Light Image Enhancement with Color and Luminance Priors [13.688097246812042]
  Low-light images often suffer from low contrast, noise, and color distortion, degrading visual quality and impairing downstream vision tasks.<n>We propose a novel conditional diffusion framework for low-light image enhancement that incorporates a Structured Control Embedding Module (SCEM)<n>SCEM decomposes a low-light image into four informative components including illumination, illumination-invariant features, shadow priors, and color-invariant cues.
  arXiv  Detail & Related papers  (2026-02-27T22:15:27Z)
• LightQANet: Quantized and Adaptive Feature Learning for Low-Light Image Enhancement [65.06462316546806]
  Low-light image enhancement aims to improve illumination while preserving high-quality color and texture.<n>Existing methods often fail to extract reliable feature representations due to severely degraded pixel-level information under low-light conditions.<n>We propose LightQANet, a novel framework that introduces quantized and adaptive feature learning for low-light enhancement.
  arXiv  Detail & Related papers  (2025-10-16T14:54:42Z)
• PractiLight: Practical Light Control Using Foundational Diffusion Models [78.75949075070595]
  PractiLight is a practical approach to light control in generated images.<n>Our key insight is that lighting relationships in an image are similar in nature to token interaction in self-attention layers.<n>We demonstrate state-of-the-art performance in terms of quality and control with proven parameter and data efficiency.
  arXiv  Detail & Related papers  (2025-09-01T23:38:40Z)
• SAIGFormer: A Spatially-Adaptive Illumination-Guided Network for Low-Light Image Enhancement [58.79901582809091]
  Recent Transformer-based low-light enhancement methods have made promising progress in recovering global illumination.<n>Recent Transformer-based low-light enhancement methods have made promising progress in recovering global illumination.<n>We present a Spatially-Adaptive Illumination-Guided Transformer framework that enables accurate illumination restoration.
  arXiv  Detail & Related papers  (2025-07-21T11:38:56Z)
• LightLab: Controlling Light Sources in Images with Diffusion Models [49.83835236202516]
  We present a diffusion-based method for fine-grained, parametric control over light sources in an image.<n>We leverage the linearity of light to synthesize image pairs depicting controlled light changes of either a target light source or ambient illumination.<n>We show how our method can achieve compelling light editing results, and outperforms existing methods based on user preference.
  arXiv  Detail & Related papers  (2025-05-14T17:57:27Z)
• LumiNet: Latent Intrinsics Meets Diffusion Models for Indoor Scene Relighting [26.40653597095593]
  Given a source image and a target lighting image, LumiNet synthesizes a relit version of the source scene that captures the target's lighting.<n>LumiNet processes latent representations from two different images - preserving geometry and albedo from the source while transferring lighting characteristics from the target.
  arXiv  Detail & Related papers  (2024-11-29T18:59:11Z)
• DifFRelight: Diffusion-Based Facial Performance Relighting [12.909429637057343]
  We present a novel framework for free-viewpoint facial performance relighting using diffusion-based image-to-image translation. We train a diffusion model for precise lighting control, enabling high-fidelity relit facial images from flat-lit inputs. The model accurately reproduces complex lighting effects like eye reflections, subsurface scattering, self-shadowing, and translucency.
  arXiv  Detail & Related papers  (2024-10-10T17:56:44Z)
• Self-correcting LLM-controlled Diffusion Models [83.26605445217334]
  We introduce Self-correcting LLM-controlled Diffusion (SLD) SLD is a framework that generates an image from the input prompt, assesses its alignment with the prompt, and performs self-corrections on the inaccuracies in the generated image. Our approach can rectify a majority of incorrect generations, particularly in generative numeracy, attribute binding, and spatial relationships.
  arXiv  Detail & Related papers  (2023-11-27T18:56:37Z)
• Dimma: Semi-supervised Low Light Image Enhancement with Adaptive Dimming [0.728258471592763]
  Enhancing low-light images while maintaining natural colors is a challenging problem due to camera processing variations. We propose Dimma, a semi-supervised approach that aligns with any camera by utilizing a small set of image pairs. We achieve that by introducing a convolutional mixture density network that generates distorted colors of the scene based on the illumination differences.
  arXiv  Detail & Related papers  (2023-10-14T17:59:46Z)
• Unsupervised Low-light Image Enhancement with Decoupled Networks [103.74355338972123]
  We learn a two-stage GAN-based framework to enhance the real-world low-light images in a fully unsupervised fashion. Our proposed method outperforms the state-of-the-art unsupervised image enhancement methods in terms of both illumination enhancement and noise reduction.
  arXiv  Detail & Related papers  (2020-05-06T13:37:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.