LENS: Low-Frequency Eigen Noise Shaping for Efficient Diffusion Sampling

• URL: http://arxiv.org/abs/2605.07253v1
• Date: Fri, 08 May 2026 05:22:24 GMT
• Title: LENS: Low-Frequency Eigen Noise Shaping for Efficient Diffusion Sampling
• Abstract summary: Distilled diffusion models accelerate image generation by reducing the number of denoising steps, but often suffer from degraded image quality.<n>Recent hypernetwork-based approaches amortize this process during training, but still require costly noise modulation in high-dimensional latent spaces.<n>We propose LENS (Low-frequency Eigen Noise Shaping), an efficient noise modulation framework that operates in a low-dimensional subspace.
• Score: 10.887547857646942
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Distilled diffusion models accelerate image generation by reducing the number of denoising steps, but often suffer from degraded image quality. To mitigate this trade-off, test-time optimization methods improve quality, yet their iterative nature incurs substantial computational overhead and leads to slow inference, limiting practical usability. Recent hypernetwork-based approaches amortize this process during training, but still require costly noise modulation in high-dimensional latent spaces. In this work, we propose LENS (Low-frequency Eigen Noise Shaping), an efficient noise modulation framework that operates in a low-dimensional subspace. Our approach is motivated by the observation that low-frequency components of the noise largely determine the global structure and visual fidelity of generated images. Based on this observation, we provide a theoretical justification for restricting modulation to the low-frequency subspace and derive a principled training objective. Building on this, LENS employs a lightweight, standalone network to selectively modulate these components, enabling efficient and targeted noise modulation. Extensive experiments demonstrate that LENS achieves competitive image quality while reducing FLOPs by 400-700$\times$, model parameters by 25-75$\times$, and inference-time overhead by 10-20$\times$ compared to prior methods.

Related papers

• Pixel-Space Diffusion via Observation Operators [22.391778551506345]
  We propose a unified framework that aligns full-image supervision and feature refinement with the intrinsic recovery order of image structures.<n>Experiments show that the proposed approach converges substantially faster while consistently improving generation quality, achieving an FID of 1.52 on ImageNet-256.
  arXiv  Detail & Related papers  (2026-08-22T09:54:01Z)
• Class-frequency Guided Noise Schedule for Diffusion Models [83.19018650759415]
  We first examine the correlations between class frequency and the multi-scale noise schedule within diffusion models.<n>We propose the textitClass-frequency Guided (CFRG) noise schedule, leveraging the insight that low-frequency classes should be endowed with larger-scale noises.
  arXiv  Detail & Related papers  (2026-06-26T03:43:23Z)
• Spatial-Spectral Adaptive Fidelity and Noise Prior Reduction Guided Hyperspectral Image Denoising [11.513560890234864]
  We propose a framework that integrates noise prior reduction and spatial-spectral adaptive fidelity term.<n>Within this framework, we develop a fast and robust pixel-wise model combined with the representative coefficient total variation regularizer.<n>Experiments on simulated and real-world datasets demonstrate that the proposed model achieves superior denoising performance.
  arXiv  Detail & Related papers  (2026-04-14T11:27:23Z)
• Single-Stage Signal Attenuation Diffusion Model for Low-Light Image Enhancement and Denoising [4.397450561168298]
  Diffusion models excel at image restoration via probabilistic modeling of forward noise addition and reverse denoising.<n>Mainstream diffusion based Low-Light Image Enhancement (LLIE) methods either adopt a two-stage pipeline or an auxiliary correction network to refine U-Net outputs.<n>We propose the Signal Attenuation Diffusion Model (SADM), a novel diffusion process that integrates the signal attenuation mechanism into the diffusion pipeline.
  arXiv  Detail & Related papers  (2026-04-07T11:33:04Z)
• BADiff: Bandwidth Adaptive Diffusion Model [55.10134744772338]
  Traditional diffusion models produce high-fidelity images by performing a fixed number of denoising steps, regardless of downstream transmission limitations.<n>In practical cloud-to-device scenarios, limited bandwidth often necessitates heavy compression, leading to loss of fine textures and wasted computation.<n>We introduce a joint end-to-end training strategy where the diffusion model is conditioned on a target quality level derived from the available bandwidth.
  arXiv  Detail & Related papers  (2025-10-24T11:50:03Z)
• Mitigating the Noise Shift for Denoising Generative Models via Noise Awareness Guidance [54.88271057438763]
  Noise Awareness Guidance (NAG) is a correction method that explicitly steers sampling trajectories to remain consistent with the pre-defined noise schedule.<n>NAG consistently mitigates noise shift and substantially improves the generation quality of mainstream diffusion models.
  arXiv  Detail & Related papers  (2025-10-14T13:31:34Z)
• Physics-Guided Rectified Flow for Low-light RAW Image Enhancement [0.0]
  Enhancing RAW images captured under low light conditions is a challenging task.<n>Recent deep learning based RAW enhancement methods have shifted from using real paired data to relying on synthetic datasets.
  arXiv  Detail & Related papers  (2025-09-10T07:08:43Z)
• Noise Conditional Variational Score Distillation [60.38982038894823]
  Noise Conditional Variational Score Distillation (NCVSD) is a novel method for distilling pretrained diffusion models into generative denoisers.<n>By integrating this insight into the Variational Score Distillation framework, we enable scalable learning of generative denoisers.
  arXiv  Detail & Related papers  (2025-06-11T06:01:39Z)
• Dark Noise Diffusion: Noise Synthesis for Low-Light Image Denoising [22.897202020483576]
  Low-light photography produces images with low signal-to-noise ratios due to limited photons.<n>Deep-learning methods perform well, but they require large datasets of paired images that are impractical to acquire.<n>In this paper, we investigate the ability of diffusion models to capture the complex distribution of low-light noise.
  arXiv  Detail & Related papers  (2025-03-14T10:16:54Z)
• Enhanced Confocal Laser Scanning Microscopy with Adaptive Physics Informed Deep Autoencoders [0.0]
  We present a physics-informed deep learning framework to address limitations in Confocal Laser Scanning Microscopy.<n>The model reconstructs high fidelity images from heavily noisy inputs by using convolutional and transposed convolutional layers.
  arXiv  Detail & Related papers  (2025-01-24T18:32:34Z)
• Realistic Noise Synthesis with Diffusion Models [44.404059914652194]
  Deep denoising models require extensive real-world training data, which is challenging to acquire.<n>We propose a novel Realistic Noise Synthesis Diffusor (RNSD) method using diffusion models to address these challenges.
  arXiv  Detail & Related papers  (2023-05-23T12:56:01Z)
• Advancing Unsupervised Low-light Image Enhancement: Noise Estimation, Illumination Interpolation, and Self-Regulation [55.07472635587852]
  Low-Light Image Enhancement (LLIE) techniques have made notable advancements in preserving image details and enhancing contrast. These approaches encounter persistent challenges in efficiently mitigating dynamic noise and accommodating diverse low-light scenarios. We first propose a method for estimating the noise level in low light images in a quick and accurate way. We then devise a Learnable Illumination Interpolator (LII) to satisfy general constraints between illumination and input.
  arXiv  Detail & Related papers  (2023-05-17T13:56:48Z)
• Zero-shot Blind Image Denoising via Implicit Neural Representations [77.79032012459243]
  We propose an alternative denoising strategy that leverages the architectural inductive bias of implicit neural representations (INRs) We show that our method outperforms existing zero-shot denoising methods under an extensive set of low-noise or real-noise scenarios.
  arXiv  Detail & Related papers  (2022-04-05T12:46:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.