On the Anisotropy of Score-Based Generative Models

• URL: http://arxiv.org/abs/2510.22899v1
• Date: Mon, 27 Oct 2025 01:01:41 GMT
• Title: On the Anisotropy of Score-Based Generative Models
• Abstract summary: We introduce the Score Anisotropy Directions (SADs), architecture-dependent directions that reveal how different networks preferentially capture data structure.<n>Our work offers a new lens for explaining and predicting directional biases of generative models.
• Score: 21.619549942021592
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We investigate the role of network architecture in shaping the inductive biases of modern score-based generative models. To this end, we introduce the Score Anisotropy Directions (SADs), architecture-dependent directions that reveal how different networks preferentially capture data structure. Our analysis shows that SADs form adaptive bases aligned with the architecture's output geometry, providing a principled way to predict generalization ability in score models prior to training. Through both synthetic data and standard image benchmarks, we demonstrate that SADs reliably capture fine-grained model behavior and correlate with downstream performance, as measured by Wasserstein metrics. Our work offers a new lens for explaining and predicting directional biases of generative models.

Related papers

• Deep Neural Networks as Iterated Function Systems and a Generalization Bound [2.7920304852537536]
  We show that two important deep architectures can be viewed as, or canonically associated with, place-dependent IFS.<n>We derive a Wasserstein bound for generative modeling that controls the collage-type approximation error between the data distribution and its image.
  arXiv  Detail & Related papers  (2026-01-27T07:32:49Z)
• Nonparametric Data Attribution for Diffusion Models [57.820618036556084]
  Data attribution for generative models seeks to quantify the influence of individual training examples on model outputs.<n>We propose a nonparametric attribution method that operates entirely on data, measuring influence via patch-level similarity between generated and training images.
  arXiv  Detail & Related papers  (2025-10-16T03:37:16Z)
• Generalized Factor Neural Network Model for High-dimensional Regression [50.554377879576066]
  We tackle the challenges of modeling high-dimensional data sets with latent low-dimensional structures hidden within complex, non-linear, and noisy relationships.<n>Our approach enables a seamless integration of concepts from non-parametric regression, factor models, and neural networks for high-dimensional regression.
  arXiv  Detail & Related papers  (2025-02-16T23:13:55Z)
• Elucidating Flow Matching ODE Dynamics with Respect to Data Geometries and Denoisers [10.947094609205765]
  Flow matching (FM) models extend ODE sampler based diffusion models into a general framework.<n>A rigorous theoretical analysis of FM models is essential for sample quality, stability, and broader applicability.<n>In this paper, we advance the theory of FM models through a comprehensive analysis of sample trajectories.
  arXiv  Detail & Related papers  (2024-12-25T01:17:15Z)
• Optimizing Sequential Recommendation Models with Scaling Laws and Approximate Entropy [104.48511402784763]
  Performance Law for SR models aims to theoretically investigate and model the relationship between model performance and data quality.<n>We propose Approximate Entropy (ApEn) to assess data quality, presenting a more nuanced approach compared to traditional data quantity metrics.
  arXiv  Detail & Related papers  (2024-11-30T10:56:30Z)
• Sub-graph Based Diffusion Model for Link Prediction [43.15741675617231]
  Denoising Diffusion Probabilistic Models (DDPMs) represent a contemporary class of generative models with exceptional qualities. We build a novel generative model for link prediction using a dedicated design to decompose the likelihood estimation process via the Bayesian formula. Our proposed method presents numerous advantages: (1) transferability across datasets without retraining, (2) promising generalization on limited training data, and (3) robustness against graph adversarial attacks.
  arXiv  Detail & Related papers  (2024-09-13T02:23:55Z)
• PerturBench: Benchmarking Machine Learning Models for Cellular Perturbation Analysis [8.785345412061792]
  We introduce a comprehensive framework for modeling single cell transcriptomic responses to perturbations.<n>Our approach includes a modular and user-friendly model development and evaluation platform.<n>We highlight the limitations of widely used models, such as mode collapse.
  arXiv  Detail & Related papers  (2024-08-20T07:40:20Z)
• Probabilistic Traffic Forecasting with Dynamic Regression [15.31488551912888]
  This paper proposes a dynamic regression (DR) framework that enhances existing deeptemporal models by incorporating for learning the error process in traffic forecasting.<n>The framework relaxes the assumption of time independence by modeling the error series of the base model using a matrix- structured autoregressive (AR) model.<n>The newly designed loss function is based on the likelihood of a non-isotropic error term, enabling the model to generate probabilistic forecasts while preserving the original outputs of the base model.
  arXiv  Detail & Related papers  (2023-01-17T01:12:44Z)
• Device Modeling Bias in ReRAM-based Neural Network Simulations [1.5490932775843136]
  Data-driven modeling approaches such as jump tables are promising to model memory devices for neural network simulations. We study how various jump table device models impact the attained network performance estimates. Results on a multi-layer perceptron trained on MNIST show that device models based on binning can behave unpredictably.
  arXiv  Detail & Related papers  (2022-11-29T04:45:06Z)
• Sparse Flows: Pruning Continuous-depth Models [107.98191032466544]
  We show that pruning improves generalization for neural ODEs in generative modeling. We also show that pruning finds minimal and efficient neural ODE representations with up to 98% less parameters compared to the original network, without loss of accuracy.
  arXiv  Detail & Related papers  (2021-06-24T01:40:17Z)
• Self-Learning for Received Signal Strength Map Reconstruction with Neural Architecture Search [63.39818029362661]
  We present a model based on Neural Architecture Search (NAS) and self-learning for received signal strength ( RSS) map reconstruction. The approach first finds an optimal NN architecture and simultaneously train the deduced model over some ground-truth measurements of a given ( RSS) map. Experimental results show that signal predictions of this second model outperforms non-learning based state-of-the-art techniques and NN models with no architecture search.
  arXiv  Detail & Related papers  (2021-05-17T12:19:22Z)
• GELATO: Geometrically Enriched Latent Model for Offline Reinforcement Learning [54.291331971813364]
  offline reinforcement learning approaches can be divided into proximal and uncertainty-aware methods. In this work, we demonstrate the benefit of combining the two in a latent variational model. Our proposed metrics measure both the quality of out of distribution samples as well as the discrepancy of examples in the data.
  arXiv  Detail & Related papers  (2021-02-22T19:42:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.