Correlation Analysis of Generative Models

• URL: http://arxiv.org/abs/2603.06614v1
• Date: Fri, 20 Feb 2026 06:28:08 GMT
• Title: Correlation Analysis of Generative Models
• Abstract summary: Our theoretical analysis shows that the correlation between the noisy data and the predicted target is sometimes weak in the existing diffusion models and flow matching.<n>This might affect the prediction (or learning) process which plays a crucial role in all models.
• Score: 17.52682596337417
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Based on literature review about existing diffusion models and flow matching with a neural network to predict a predefined target from noisy data, a unified representation is first proposed for these models using two simple linear equations in this paper. Theoretical analysis of the proposed model is then presented. Our theoretical analysis shows that the correlation between the noisy data and the predicted target is sometimes weak in the existing diffusion models and flow matching. This might affect the prediction (or learning) process which plays a crucial role in all models.

Related papers

• Nonparametric undirected graphical model selection using diffusion models [3.6515314438258097]
  We develop a nonparametric approach to undirected graphical model selection based on diffusion models.<n>We demonstrate its empirical performance through extensive simulations and two real data analyses.
  arXiv  Detail & Related papers  (2026-06-07T06:22:13Z)
• Diffusion Model for Manifold Data: Score Decomposition, Curvature, and Statistical Complexity [93.52679474816728]
  Diffusion models have become a leading framework in generative modeling.<n>This paper investigates how diffusion models learn structured data, focusing on two key aspects: statistical complexity and influence of data geometric properties.<n>We also highlight the interplay of manifold curvature with the structures in the score function.
  arXiv  Detail & Related papers  (2026-03-21T04:40:19Z)
• Improved Sample Complexity For Diffusion Model Training Without Empirical Risk Minimizer Access [47.96419637803502]
  We present a principled theoretical framework analyzing diffusion models, providing a state-of-the-art sample complexity bound to $widetildemathcalO(epsilon-4)$.<n>Our structured decomposition of the score estimation error into statistical and optimization components offers critical insights into how diffusion models can be trained efficiently.
  arXiv  Detail & Related papers  (2025-05-23T20:02:15Z)
• Elucidating Flow Matching ODE Dynamics with Respect to Data Geometries and Denoisers [10.947094609205765]
  Flow matching (FM) models extend ODE sampler based diffusion models into a general framework.<n>A rigorous theoretical analysis of FM models is essential for sample quality, stability, and broader applicability.<n>In this paper, we advance the theory of FM models through a comprehensive analysis of sample trajectories.
  arXiv  Detail & Related papers  (2024-12-25T01:17:15Z)
• Modeling and Discovering Direct Causes for Predictive Models [0.0]
  We introduce a causal modeling framework that captures the input-output behavior of predictive models.<n>We then present sound and complete algorithms for discovering direct causes (from data) under some assumptions.
  arXiv  Detail & Related papers  (2024-12-03T22:25:42Z)
• On the Feature Learning in Diffusion Models [26.53807235141923]
  We propose a feature learning framework aimed at analyzing and comparing the training dynamics of diffusion models with those of traditional classification models.<n>Our theoretical analysis demonstrates that diffusion models, due to the denoising objective, are encouraged to learn more balanced and comprehensive representations of the data.<n>In contrast, neural networks with a similar architecture trained for classification tend to prioritize learning specific patterns in the data, often focusing on easy-to-learn components.
  arXiv  Detail & Related papers  (2024-12-02T00:41:25Z)
• Towards Theoretical Understandings of Self-Consuming Generative Models [56.84592466204185]
  This paper tackles the emerging challenge of training generative models within a self-consuming loop. We construct a theoretical framework to rigorously evaluate how this training procedure impacts the data distributions learned by future models. We present results for kernel density estimation, delivering nuanced insights such as the impact of mixed data training on error propagation.
  arXiv  Detail & Related papers  (2024-02-19T02:08:09Z)
• Analyzing Neural Network-Based Generative Diffusion Models through Convex Optimization [45.72323731094864]
  We present a theoretical framework to analyze two-layer neural network-based diffusion models. We prove that training shallow neural networks for score prediction can be done by solving a single convex program. Our results provide a precise characterization of what neural network-based diffusion models learn in non-asymptotic settings.
  arXiv  Detail & Related papers  (2024-02-03T00:20:25Z)
• A PAC-Bayesian Perspective on the Interpolating Information Criterion [54.548058449535155]
  We show how a PAC-Bayes bound is obtained for a general class of models, characterizing factors which influence performance in the interpolating regime. We quantify how the test error for overparameterized models achieving effectively zero training error depends on the quality of the implicit regularization imposed by e.g. the combination of model, parameter-initialization scheme.
  arXiv  Detail & Related papers  (2023-11-13T01:48:08Z)
• How Much is Enough? A Study on Diffusion Times in Score-based Generative Models [76.76860707897413]
  Current best practice advocates for a large T to ensure that the forward dynamics brings the diffusion sufficiently close to a known and simple noise distribution. We show how an auxiliary model can be used to bridge the gap between the ideal and the simulated forward dynamics, followed by a standard reverse diffusion process.
  arXiv  Detail & Related papers  (2022-06-10T15:09:46Z)
• On the Generalization and Adaption Performance of Causal Models [99.64022680811281]
  Differentiable causal discovery has proposed to factorize the data generating process into a set of modules. We study the generalization and adaption performance of such modular neural causal models. Our analysis shows that the modular neural causal models outperform other models on both zero and few-shot adaptation in low data regimes.
  arXiv  Detail & Related papers  (2022-06-09T17:12:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.