Where the Score Lives: A Wavelet View of Diffusion

• URL: http://arxiv.org/abs/2606.08309v1
• Date: Sat, 06 Jun 2026 19:41:56 GMT
• Title: Where the Score Lives: A Wavelet View of Diffusion
• Abstract summary: We propose an analytically solvable parameterization of the score function using an expansion in a 2D wavelet basis.<n>We derive interpretable optimal score functions in terms of the moments of the data distribution.<n>Our score machine is flexible enough to partially mimic the relevant inductive biases of multiple architectures.
• Score: 7.668216032034199
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Score-based generative models have had remarkable success over the last decade in generating a diverse set of visually plausible images. A variety of architectures including CNNs, U-Nets, and Transformers have been used as the score-approximation network in such diffusion modeling; however, to date, relatively little is known about how these architectural choices impact generative behavior. In this work, to provide insight into this area, we propose an analytically solvable parameterization of the score function using an expansion in a 2D orthogonal wavelet basis. In particular, we derive interpretable optimal score functions in terms of the moments of the data distribution. We use this parametrization to provide an architecture-agnostic, moment-based analysis that reveals which attributes of the data distribution tend to matter most for denoising. Our score machine is flexible enough to partially mimic the relevant inductive biases of multiple architectures, including U-Nets, and CNNs, taking a step towards understanding why different score architectures can exhibit distinct generative behavior. Since our score is solvable in terms of the moments of the data, we can begin to understand how the data distribution interacts with the score network to produce the behavior we observe in diffusion models.

Related papers

• Agentic Insight Generation in VSM Simulations [4.626393563516877]
  We propose a decoupled, two-step agentic architecture for value stream map simulations.<n>By separating orchestration from data analysis, the system leverages progressive data discovery infused with domain expert knowledge.<n>Results from multiple state-of-the-art large language models demonstrate the framework's viability.
  arXiv  Detail & Related papers  (2026-04-14T08:11:53Z)
• Diffusion Model for Manifold Data: Score Decomposition, Curvature, and Statistical Complexity [93.52679474816728]
  Diffusion models have become a leading framework in generative modeling.<n>This paper investigates how diffusion models learn structured data, focusing on two key aspects: statistical complexity and influence of data geometric properties.<n>We also highlight the interplay of manifold curvature with the structures in the score function.
  arXiv  Detail & Related papers  (2026-03-21T04:40:19Z)
• Dimension-free Score Matching and Time Bootstrapping for Diffusion Models [19.62665684173391]
  Diffusion models generate samples by estimating the score function of the target distribution at various noise levels.<n>We introduce a martingale-based error decomposition and sharp variance bounds, enabling efficient learning from dependent data.<n>Building on these insights, we propose Bootstrapped Score Matching (BSM), a variance reduction technique that leverages previously learned scores to improve accuracy at higher noise levels.
  arXiv  Detail & Related papers  (2025-02-14T18:32:22Z)
• Sub-graph Based Diffusion Model for Link Prediction [43.15741675617231]
  Denoising Diffusion Probabilistic Models (DDPMs) represent a contemporary class of generative models with exceptional qualities. We build a novel generative model for link prediction using a dedicated design to decompose the likelihood estimation process via the Bayesian formula. Our proposed method presents numerous advantages: (1) transferability across datasets without retraining, (2) promising generalization on limited training data, and (3) robustness against graph adversarial attacks.
  arXiv  Detail & Related papers  (2024-09-13T02:23:55Z)
• ChiroDiff: Modelling chirographic data with Diffusion Models [132.5223191478268]
  We introduce a powerful model-class namely "Denoising Diffusion Probabilistic Models" or DDPMs for chirographic data. Our model named "ChiroDiff", being non-autoregressive, learns to capture holistic concepts and therefore remains resilient to higher temporal sampling rate.
  arXiv  Detail & Related papers  (2023-04-07T15:17:48Z)
• Revisiting the Evaluation of Image Synthesis with GANs [55.72247435112475]
  This study presents an empirical investigation into the evaluation of synthesis performance, with generative adversarial networks (GANs) as a representative of generative models. In particular, we make in-depth analyses of various factors, including how to represent a data point in the representation space, how to calculate a fair distance using selected samples, and how many instances to use from each set.
  arXiv  Detail & Related papers  (2023-04-04T17:54:32Z)
• Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data [68.62134204367668]
  This paper studies score approximation, estimation, and distribution recovery of diffusion models, when data are supported on an unknown low-dimensional linear subspace. We show that with a properly chosen neural network architecture, the score function can be both accurately approximated and efficiently estimated. The generated distribution based on the estimated score function captures the data geometric structures and converges to a close vicinity of the data distribution.
  arXiv  Detail & Related papers  (2023-02-14T17:02:35Z)
• A graph representation based on fluid diffusion model for multimodal data analysis: theoretical aspects and enhanced community detection [14.601444144225875]
  We introduce a novel model for graph definition based on fluid diffusion. Our method is able to strongly outperform state-of-the-art schemes for community detection in multimodal data analysis.
  arXiv  Detail & Related papers  (2021-12-07T16:30:03Z)
• Probabilistic Tracking with Deep Factors [8.030212474745879]
  We show how to use a deep feature encoding in conjunction with generative densities over the features in a factor-graph based, probabilistic tracking framework. We present a likelihood model that combines a learned feature encoder with generative densities over them, both trained in a supervised manner.
  arXiv  Detail & Related papers  (2021-12-02T21:31:51Z)
• Score-informed Networks for Music Performance Assessment [64.12728872707446]
  Deep neural network-based methods incorporating score information into MPA models have not yet been investigated. We introduce three different models capable of score-informed performance assessment.
  arXiv  Detail & Related papers  (2020-08-01T07:46:24Z)
• Out-of-distribution Generalization via Partial Feature Decorrelation [72.96261704851683]
  We present a novel Partial Feature Decorrelation Learning (PFDL) algorithm, which jointly optimize a feature decomposition network and the target image classification model. The experiments on real-world datasets demonstrate that our method can improve the backbone model's accuracy on OOD image classification datasets.
  arXiv  Detail & Related papers  (2020-07-30T05:48:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.