Diffusion Models for High-Dimensional Clustered Data: Intrinsic-Dimension Adaptivity via Bayesian Classification

• URL: http://arxiv.org/abs/2608.19067v1
• Date: Wed, 19 Aug 2026 16:09:31 GMT
• Title: Diffusion Models for High-Dimensional Clustered Data: Intrinsic-Dimension Adaptivity via Bayesian Classification
• Abstract summary: We study the adaptivity of multimodal diffusion models to structured geometry of high-dimensional data.<n>We employ $K$-mixture Gaussian distributions as a canonical framework to capture this geometry.<n>By separately analysing the denoising process in its mixing and cluster-commitment phases, we prove that the KL error bound depends linearly on the maximum intrinsic dimension of a cluster.
• Score: 1.9981375888949475
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The empirical success of diffusion models in generative modelling has motivated theoretical work, including quantitative error bounds and qualitative analyses that characterise the different phases of denoising. We bring these two areas together by studying the adaptivity of diffusion models to the structured geometry of multimodal high-dimensional data that consists of multiple clusters in $\mathbb{R}^D$, each with its own low-dimensional structure, and inter-cluster separation depending on $D$. We employ $K$-mixture Gaussian distributions as a canonical framework to capture this geometry and establish two theoretical results. First, we interpret denoising as a dynamical Bayesian classifier: the mixture score is a posterior-weighted average of cluster-wise scores, and we show that, with high probability, the posterior class probabilities concentrate on a single cluster once the signal-to-noise ratio reaches the scale $Θ(\log (KD)/D)$. Second, by separately analysing the denoising process in its mixing and cluster-commitment phases, we prove that the KL error bound depends linearly on the maximum intrinsic dimension of a cluster, up to a logarithmic factor, even when $K$ grows polynomially with $D$. This improves on ambient-dimensional bounds and extends existing low-dimensional adaptivity analyses to multimodal distributions with heterogeneous, approximately low-rank covariances.

Related papers

• Multi-kernel spectral clustering: Entrywise eigenvector perturbation bounds and exact recovery [5.506184984859199]
  Kernel spectral clustering with a single bandwidth can be inadequate for data exhibiting multiple characteristic pairwise-distance scales.<n>We address this issue through a multi- kernel formulation that aggregates kernels with different bandwidths.<n>We develop a rigorous theoretical analysis of the resulting method under a general high-dimensional, multi-scale mixture model.
  arXiv  Detail & Related papers  (2026-08-09T13:34:27Z)
• Provably Learning Diffusion Models under the Manifold Hypothesis: Collapse and Refine [60.669081685261965]
  Diffusion models generate high-dimensional data with remarkable quality.<n>How their training efficiently learns the score function remains theoretically unexplained.<n>We instantiate this principle as Score-induced Latent Diffusion (SiLD)
  arXiv  Detail & Related papers  (2026-05-16T16:51:10Z)
• Efficient Sampling with Discrete Diffusion Models: Sharp and Adaptive Guarantees [9.180350432640912]
  We study the sampling efficiency of score-based discrete diffusion models under a continuous-time Markov chain (CTMC) formulation.<n>For uniform discrete diffusion, we show that the $$-leaping algorithm achieves an complexity of order $tilde O(d/varepsilon)$.<n>For masking discrete diffusion, we introduce a modified $$-leaping sampler whose convergence rate is governed by an intrinsic information-theoretic quantity.
  arXiv  Detail & Related papers  (2026-02-16T18:48:17Z)
• Minimax-Optimal Dimension-Reduced Clustering for High-Dimensional Nonspherical Mixtures [1.4502611532302039]
  Nonspherical (anisotropic) noise within each cluster is widely present in real-world data.<n>We study both the minimax rate and optimal statistical procedure for clustering under high-dimensional nonspherical mixture models.
  arXiv  Detail & Related papers  (2025-02-04T18:55:49Z)
• Consistent spectral clustering in sparse tensor block models [0.0]
  High-order clustering aims to classify objects in multiway datasets that are prevalent in various fields.<n>This paper introduces a tensor block model specifically designed for sparse integer-valued data tensors.<n>We propose a simple spectral clustering algorithm augmented with a trimming step to mitigate noise fluctuations.
  arXiv  Detail & Related papers  (2025-01-23T16:41:19Z)
• Convergence of Score-Based Discrete Diffusion Models: A Discrete-Time Analysis [56.442307356162864]
  We study the theoretical aspects of score-based discrete diffusion models under the Continuous Time Markov Chain (CTMC) framework.<n>We introduce a discrete-time sampling algorithm in the general state space $[S]d$ that utilizes score estimators at predefined time points.<n>Our convergence analysis employs a Girsanov-based method and establishes key properties of the discrete score function.
  arXiv  Detail & Related papers  (2024-10-03T09:07:13Z)
• Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models [6.76974373198208]
  We find that the dependency of the error incurred within each denoising step on the ambient dimension $d$ is in general unavoidable.<n>This represents the first theoretical demonstration that the DDPM sampler can adapt to unknown low-dimensional structures in the target distribution.
  arXiv  Detail & Related papers  (2024-05-23T17:59:10Z)
• A Geometric Perspective on Diffusion Models [57.27857591493788]
  We inspect the ODE-based sampling of a popular variance-exploding SDE. We establish a theoretical relationship between the optimal ODE-based sampling and the classic mean-shift (mode-seeking) algorithm.
  arXiv  Detail & Related papers  (2023-05-31T15:33:16Z)
• Diffusion models as plug-and-play priors [98.16404662526101]
  We consider the problem of inferring high-dimensional data $mathbfx$ in a model that consists of a prior $p(mathbfx)$ and an auxiliary constraint $c(mathbfx,mathbfy)$. The structure of diffusion models allows us to perform approximate inference by iterating differentiation through the fixed denoising network enriched with different amounts of noise.
  arXiv  Detail & Related papers  (2022-06-17T21:11:36Z)
• Learning Gaussian Mixtures with Generalised Linear Models: Precise Asymptotics in High-dimensions [79.35722941720734]
  Generalised linear models for multi-class classification problems are one of the fundamental building blocks of modern machine learning tasks. We prove exacts characterising the estimator in high-dimensions via empirical risk minimisation. We discuss how our theory can be applied beyond the scope of synthetic data.
  arXiv  Detail & Related papers  (2021-06-07T16:53:56Z)
• Kernel and Rich Regimes in Overparametrized Models [69.40899443842443]
  We show that gradient descent on overparametrized multilayer networks can induce rich implicit biases that are not RKHS norms. We also demonstrate this transition empirically for more complex matrix factorization models and multilayer non-linear networks.
  arXiv  Detail & Related papers  (2020-02-20T15:43:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.