Unleashing High-Quality Image Generation in Diffusion Sampling Using Second-Order Levenberg-Marquardt-Langevin

• URL: http://arxiv.org/abs/2505.24222v1
• Date: Fri, 30 May 2025 05:21:44 GMT
• Title: Unleashing High-Quality Image Generation in Diffusion Sampling Using Second-Order Levenberg-Marquardt-Langevin
• Authors: Fangyikang Wang, Hubery Yin, Lei Qian, Yinan Li, Shaobin Zhuang, Huminhao Zhu, Yilin Zhang, Yanlong Tang, Chao Zhang, Hanbin Zhao, Hui Qian, Chen Li,
• Abstract summary: We introduce a novel Levenberg-Marquardt-Langevin (LML) method that approximates the diffusion Hessian geometry in a training-free manner.<n>This LML approximated Hessian geometry enables the diffusion sampling to execute more accurate steps and improve the image generation quality.
• Score: 19.316680628326406
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The diffusion models (DMs) have demonstrated the remarkable capability of generating images via learning the noised score function of data distribution. Current DM sampling techniques typically rely on first-order Langevin dynamics at each noise level, with efforts concentrated on refining inter-level denoising strategies. While leveraging additional second-order Hessian geometry to enhance the sampling quality of Langevin is a common practice in Markov chain Monte Carlo (MCMC), the naive attempts to utilize Hessian geometry in high-dimensional DMs lead to quadratic-complexity computational costs, rendering them non-scalable. In this work, we introduce a novel Levenberg-Marquardt-Langevin (LML) method that approximates the diffusion Hessian geometry in a training-free manner, drawing inspiration from the celebrated Levenberg-Marquardt optimization algorithm. Our approach introduces two key innovations: (1) A low-rank approximation of the diffusion Hessian, leveraging the DMs' inherent structure and circumventing explicit quadratic-complexity computations; (2) A damping mechanism to stabilize the approximated Hessian. This LML approximated Hessian geometry enables the diffusion sampling to execute more accurate steps and improve the image generation quality. We further conduct a theoretical analysis to substantiate the approximation error bound of low-rank approximation and the convergence property of the damping mechanism. Extensive experiments across multiple pretrained DMs validate that the LML method significantly improves image generation quality, with negligible computational overhead.

Related papers

• Learning few-step posterior samplers by unfolding and distillation of diffusion models [0.18434042562191813]
  Diffusion models (DMs) have emerged as powerful image priors in computational imaging.<n>We introduce a novel framework that integrates deep unfolding and model distillation to transform a DM image prior into a few-step conditional model for posterior sampling.
  arXiv  Detail & Related papers  (2025-07-03T14:55:53Z)
• Language-Informed Hyperspectral Image Synthesis for Imbalanced-Small Sample Classification via Semi-Supervised Conditional Diffusion Model [1.9746060146273674]
  This paper proposes Txt2HSI-LDM(VAE), a novel language-informed hyperspectral image synthesis method.<n>To address the high-dimensionality of hyperspectral data, a universal variational autoencoder (VAE) is designed to map the data into a low-dimensional latent space.<n>VAE decodes HSI from latent space generated by the diffusion model with the language conditions as input.
  arXiv  Detail & Related papers  (2025-02-27T02:35:49Z)
• Empirical Bayesian image restoration by Langevin sampling with a denoising diffusion implicit prior [0.18434042562191813]
  This paper presents a novel and highly computationally efficient image restoration method. It embeds a DDPM denoiser within an empirical Bayesian Langevin algorithm. It improves on state-of-the-art strategies both in image estimation accuracy and computing time.
  arXiv  Detail & Related papers  (2024-09-06T16:20:24Z)
• The Poisson Midpoint Method for Langevin Dynamics: Provably Efficient Discretization for Diffusion Models [9.392691963008385]
  Langevin Monte Carlo (LMC) is the simplest and most studied algorithm. We propose the Poisson Midpoint Method, which approximates a small step-size LMC with large step-sizes. We show that it maintains the quality of DDPM with 1000 neural network calls with just 50-80 neural network calls and outperforms ODE based methods with similar compute.
  arXiv  Detail & Related papers  (2024-05-27T11:40:42Z)
• Proximal Algorithms for Accelerated Langevin Dynamics [57.08271964961975]
  We develop a novel class of MCMC algorithms based on a stochastized Nesterov scheme. We show superior performance of the proposed method over typical Langevin samplers for different models in statistics and image processing.
  arXiv  Detail & Related papers  (2023-11-24T19:56:01Z)
• Generative Fractional Diffusion Models [53.36835573822926]
  We introduce the first continuous-time score-based generative model that leverages fractional diffusion processes for its underlying dynamics. Our evaluations on real image datasets demonstrate that GFDM achieves greater pixel-wise diversity and enhanced image quality, as indicated by a lower FID.
  arXiv  Detail & Related papers  (2023-10-26T17:53:24Z)
• DiffuSeq-v2: Bridging Discrete and Continuous Text Spaces for Accelerated Seq2Seq Diffusion Models [58.450152413700586]
  We introduce a soft absorbing state that facilitates the diffusion model in learning to reconstruct discrete mutations based on the underlying Gaussian space. We employ state-of-the-art ODE solvers within the continuous space to expedite the sampling process. Our proposed method effectively accelerates the training convergence by 4x and generates samples of similar quality 800x faster.
  arXiv  Detail & Related papers  (2023-10-09T15:29:10Z)
• Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference [60.32804641276217]
  We propose Latent Consistency Models (LCMs), enabling swift inference with minimal steps on any pre-trained LDMs. A high-quality 768 x 768 24-step LCM takes only 32 A100 GPU hours for training. We also introduce Latent Consistency Fine-tuning (LCF), a novel method that is tailored for fine-tuning LCMs on customized image datasets.
  arXiv  Detail & Related papers  (2023-10-06T17:11:58Z)
• A Geometric Perspective on Diffusion Models [57.27857591493788]
  We inspect the ODE-based sampling of a popular variance-exploding SDE. We establish a theoretical relationship between the optimal ODE-based sampling and the classic mean-shift (mode-seeking) algorithm.
  arXiv  Detail & Related papers  (2023-05-31T15:33:16Z)
• Hierarchical Integration Diffusion Model for Realistic Image Deblurring [71.76410266003917]
  Diffusion models (DMs) have been introduced in image deblurring and exhibited promising performance. We propose the Hierarchical Integration Diffusion Model (HI-Diff), for realistic image deblurring. Experiments on synthetic and real-world blur datasets demonstrate that our HI-Diff outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-22T12:18:20Z)
• Sampling in Combinatorial Spaces with SurVAE Flow Augmented MCMC [83.48593305367523]
  Hybrid Monte Carlo is a powerful Markov Chain Monte Carlo method for sampling from complex continuous distributions. We introduce a new approach based on augmenting Monte Carlo methods with SurVAE Flows to sample from discrete distributions. We demonstrate the efficacy of our algorithm on a range of examples from statistics, computational physics and machine learning, and observe improvements compared to alternative algorithms.
  arXiv  Detail & Related papers  (2021-02-04T02:21:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.