Generative Myopia: Why Diffusion Models Fail at Structure

• URL: http://arxiv.org/abs/2511.18593v1
• Date: Sun, 23 Nov 2025 19:27:13 GMT
• Title: Generative Myopia: Why Diffusion Models Fail at Structure
• Authors: Milad Siami,
• Abstract summary: Graph Diffusion Models (GDMs) optimize for statistical likelihood, implicitly acting as textbffrequency filtersGenerative Myopia.<n>We prove theoretically and empirically that this failure is driven by textbfGradient Starvation.<n>We introduce textbfSpectrally-Weighted Diffusion, which re-aligns the variational objective using Effective Resistance.
• Score: 0.6768558752130311
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Graph Diffusion Models (GDMs) optimize for statistical likelihood, implicitly acting as \textbf{frequency filters} that favor abundant substructures over spectrally critical ones. We term this phenomenon \textbf{Generative Myopia}. In combinatorial tasks like graph sparsification, this leads to the catastrophic removal of ``rare bridges,'' edges that are structurally mandatory ($R_{\text{eff}} \approx 1$) but statistically scarce. We prove theoretically and empirically that this failure is driven by \textbf{Gradient Starvation}: the optimization landscape itself suppresses rare structural signals, rendering them unlearnable regardless of model capacity. To resolve this, we introduce \textbf{Spectrally-Weighted Diffusion}, which re-aligns the variational objective using Effective Resistance. We demonstrate that spectral priors can be amortized into the training phase with zero inference overhead. Our method eliminates myopia, matching the performance of an optimal Spectral Oracle and achieving \textbf{100\% connectivity} on adversarial benchmarks where standard diffusion fails completely (0\%).

Related papers

• Spectral Collapse in Diffusion Inversion [44.781674986581244]
  Conditional diffusion inversion fails when the source domain is spectrally sparse compared to the target domain.<n>We propose Orthogonal Variance Guidance (OVG), an inference-time method that corrects the ODE dynamics to enforce the theoretical Gaussian noise magnitude.<n>OVG effectively restores photorealistic textures while preserving structural fidelity.
  arXiv  Detail & Related papers  (2026-02-09T17:53:21Z)
• Hessian Spectral Analysis at Foundation Model Scale [1.9244735303181757]
  We show that faithful spectral analysis of the true Hessian is tractable at frontier scale.<n>We produce the first large-scale spectral density estimates beyond the sub-10B regime.
  arXiv  Detail & Related papers  (2026-01-31T16:57:06Z)
• Latent Sculpting for Zero-Shot Generalization: A Manifold Learning Approach to Out-of-Distribution Anomaly Detection [2.8547732086436306]
  A fundamental limitation of supervised deep learning is "Generalization Collapse"<n>We propose Latent Sculpting, a hierarchical two-stage representation learning framework.<n>We report an 88.89% detection rate on "Infiltration" scenarios.
  arXiv  Detail & Related papers  (2025-12-19T11:37:02Z)
• Sparse Attention Post-Training for Mechanistic Interpretability [55.030850996535776]
  We introduce a simple post-training method that makes transformer attention sparse without sacrificing performance.<n>Applying a flexible sparsity regularisation under a constrained-loss objective, we show on models up to 1B parameters that it is possible to retain the original pretraining loss while reducing attention connectivity to $approx 0.3 % of its edges.
  arXiv  Detail & Related papers  (2025-12-05T16:40:08Z)
• HybSpecNet: A Critical Analysis of Architectural Instability in Hybrid-Domain Spectral GNNs [0.0]
  Spectral Graph Neural Networks offer a principled approach to graph filtering.<n>They face a fundamental "Stability-vs.Adaptivity" trade-off.<n>We propose a hybrid-domain GNN, HybSpecNet, which combines a stable ChebyNet branch with an adaptive KrawtchoukNet branch.
  arXiv  Detail & Related papers  (2025-11-20T06:47:44Z)
• Learning What Matters: Steering Diffusion via Spectrally Anisotropic Forward Noise [43.07594740645669]
  Diffusion Probabilistic Models (DPMs) have achieved strong generative performance, yet their inductive biases remain largely implicit.<n>In this work, we aim to build inductive biases into the training and sampling of diffusion models to better accommodate the target distribution of the data to model.<n>We introduce an anisotropic noise operator that shapes these biases by replacing the isotropic forward covariance with a structured, frequency-diagonal covariance.
  arXiv  Detail & Related papers  (2025-10-07T16:08:39Z)
• Graph-based Clustering Revisited: A Relaxation of Kernel $k$-Means Perspective [73.18641268511318]
  We propose a graph-based clustering algorithm that only relaxes the orthonormal constraint to derive clustering results.<n>To ensure a doubly constraint into a gradient, we transform the non-negative constraint into a class probability parameter.
  arXiv  Detail & Related papers  (2025-09-23T09:14:39Z)
• LLM Probability Concentration: How Alignment Shrinks the Generative Horizon [13.184240238106016]
  We show that alignment tuning substantially sharpens the model's output distribution from the outset.<n>Building on this insight, we find this consistency has surprising implications for complex reasoning.
  arXiv  Detail & Related papers  (2025-06-22T02:00:37Z)
• Generalized Interpolating Discrete Diffusion [65.74168524007484]
  Masked diffusion is a popular choice due to its simplicity and effectiveness.<n>We generalize a new family of general interpolating discrete diffusion (GIDD) which offers greater flexibility in the design of the noising processes.<n>Exploiting GIDD's flexibility, we explore a hybrid approach combining masking and uniform noise, leading to improved sample quality.
  arXiv  Detail & Related papers  (2025-03-06T14:30:55Z)
• Rao-Blackwell Gradient Estimators for Equivariant Denoising Diffusion [55.95767828747407]
  In domains such as molecular and protein generation, physical systems exhibit inherent symmetries that are critical to model.<n>We present a framework that reduces training variance and provides a provably lower-variance gradient estimator.<n>We also present a practical implementation of this estimator incorporating the loss and sampling procedure through a method we call Orbit Diffusion.
  arXiv  Detail & Related papers  (2025-02-14T03:26:57Z)
• Rectified Diffusion Guidance for Conditional Generation [94.83538269086613]
  We revisit the theory behind CFG and rigorously confirm that the improper combination coefficients (textiti.e.) brings about expectation shift the generative distribution.<n>We show that our approach enjoys a textbftextitform solution given the strength.<n> Empirical evidence on real-world data demonstrate the compatibility of our design with existing state-of-the-art diffusion models.
  arXiv  Detail & Related papers  (2024-10-24T13:41:32Z)
• Amortizing intractable inference in diffusion models for vision, language, and control [89.65631572949702]
  This paper studies amortized sampling of the posterior over data, $mathbfxsim prm post(mathbfx)propto p(mathbfx)r(mathbfx)$, in a model that consists of a diffusion generative model prior $p(mathbfx)$ and a black-box constraint or function $r(mathbfx)$.<n>We prove the correctness of a data-free learning objective, relative trajectory balance, for training a diffusion model that samples from
  arXiv  Detail & Related papers  (2024-05-31T16:18:46Z)
• Closed-Loop Unsupervised Representation Disentanglement with $β$-VAE Distillation and Diffusion Probabilistic Feedback [26.935057415769876]
  Representation disentanglement may help AI fundamentally understand the real world and thus benefit both discrimination and generation tasks.<n>We propose a textbfCL-Disentanglement approach dubbed textbfCL-Dis.<n>Experiments demonstrate the superiority of CL-Dis on applications like real image manipulation and visual analysis.
  arXiv  Detail & Related papers  (2024-02-04T05:03:22Z)
• Optimization-Induced Graph Implicit Nonlinear Diffusion [64.39772634635273]
  We propose a new kind of graph convolution variants, called Graph Implicit Diffusion (GIND) GIND implicitly has access to infinite hops of neighbors while adaptively aggregating features with nonlinear diffusion to prevent over-smoothing. We show that the learned representation can be formalized as the minimizer of an explicit convex optimization objective.
  arXiv  Detail & Related papers  (2022-06-29T06:26:42Z)
• Gaussian MRF Covariance Modeling for Efficient Black-Box Adversarial Attacks [86.88061841975482]
  We study the problem of generating adversarial examples in a black-box setting, where we only have access to a zeroth order oracle. We use this setting to find fast one-step adversarial attacks, akin to a black-box version of the Fast Gradient Sign Method(FGSM) We show that the method uses fewer queries and achieves higher attack success rates than the current state of the art.
  arXiv  Detail & Related papers  (2020-10-08T18:36:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.