The Spacetime of Diffusion Models: An Information Geometry Perspective

• URL: http://arxiv.org/abs/2505.17517v2
• Date: Tue, 21 Oct 2025 10:31:15 GMT
• Title: The Spacetime of Diffusion Models: An Information Geometry Perspective
• Authors: Rafał Karczewski, Markus Heinonen, Alison Pouplin, Søren Hauberg, Vikas Garg,
• Abstract summary: We show that the standard pullback approach, utilizing the deterministic probability flow ComplementODE decoder, is fundamentally flawed.<n>We introduce a latent spacetime $z=(x_t,t)$ that indexes the family of denoising distributions $p(x_t,t)$ across all noise scales.<n>The resulting structure induces a principled Diffusion Distance Edit, where geodesics trace minimal sequences of noise and denoise edits between data.
• Score: 40.23096112113255
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a novel geometric perspective on the latent space of diffusion models. We first show that the standard pullback approach, utilizing the deterministic probability flow ODE decoder, is fundamentally flawed. It provably forces geodesics to decode as straight segments in data space, effectively ignoring any intrinsic data geometry beyond the ambient Euclidean space. Complementing this view, diffusion also admits a stochastic decoder via the reverse SDE, which enables an information geometric treatment with the Fisher-Rao metric. However, a choice of $x_T$ as the latent representation collapses this metric due to memorylessness. We address this by introducing a latent spacetime $z=(x_t,t)$ that indexes the family of denoising distributions $p(x_0 | x_t)$ across all noise scales, yielding a nontrivial geometric structure. We prove these distributions form an exponential family and derive simulation-free estimators for curve lengths, enabling efficient geodesic computation. The resulting structure induces a principled Diffusion Edit Distance, where geodesics trace minimal sequences of noise and denoise edits between data. We also demonstrate benefits for transition path sampling in molecular systems, including constrained variants such as low-variance transitions and region avoidance. Code is available at: https://github.com/rafalkarczewski/spacetime-geometry

Related papers

• Riemannian Flow Matching for Disentangled Graph Domain Adaptation [51.98961391065951]
  Graph Domain Adaptation (GDA) typically uses adversarial learning to align graph embeddings in Euclidean space.<n>DisRFM is a geometry-aware GDA framework that unifies embedding and flow-based transport.
  arXiv  Detail & Related papers  (2026-01-31T11:05:35Z)
• Phase-space entropy at acquisition reflects downstream learnability [54.4100065023873]
  We propose an acquisition-level scalar $S_mathcal B$ based on instrument-resolved phase space.<n>We show theoretically that (S_mathcal B) correctly identifies the phase-space coherence of periodic sampling.<n>$|S_mathcal B|$ consistently ranks sampling geometries and predicts downstream reconstruction/recognition difficulty emphwithout training.
  arXiv  Detail & Related papers  (2025-12-22T10:03:51Z)
• Be Tangential to Manifold: Discovering Riemannian Metric for Diffusion Models [13.063093054280946]
  Diffusion models are powerful deep generative models (DGMs) that generate high-fidelity, diverse content.<n>Unlike classical DGMs, they lack an explicit, tractable low-dimensional latent space that parameterizes the data manifold.<n>We propose a novel metric on the noise space, inspired by recent findings that the Jacobian of the score function captures the tangent spaces to the local data manifold.
  arXiv  Detail & Related papers  (2025-10-07T01:54:47Z)
• Enforcing Latent Euclidean Geometry in Single-Cell VAEs for Manifold Interpolation [79.27003481818413]
  We introduce FlatVI, a training framework that regularises the latent manifold of discrete-likelihood variational autoencoders towards Euclidean geometry.<n>By encouraging straight lines in the latent space to approximate geodesics on the decoded single-cell manifold, FlatVI enhances compatibility with downstream approaches.
  arXiv  Detail & Related papers  (2025-07-15T23:08:14Z)
• What's Inside Your Diffusion Model? A Score-Based Riemannian Metric to Explore the Data Manifold [0.053713376045563095]
  We introduce a score-based Riemannian metric to characterize the intrinsic geometry of a data manifold.<n>Our approach creates a geometry where geodesics naturally follow the manifold's contours.<n>We show that our score-based geodesics capture meaningful perpendicular transformations that respect the underlying data distribution.
  arXiv  Detail & Related papers  (2025-05-16T11:19:57Z)
• Generative Learning for Slow Manifolds and Bifurcation Diagrams [0.35587965024910395]
  Conditional score-based generative models (cSGMs) have demonstrated capabilities in generating plausible data from target distributions conditioned on some given label.<n>We present a framework for using cSGMs to quickly initialize on a low-dimensional (reduced-order) slow manifold of a multi-time-scale system.<n>This conditional sampling can help uncover the geometry of the reduced slow-manifold and/or approximately fill in'' missing segments of steady states in a bifurcation diagram.
  arXiv  Detail & Related papers  (2025-04-29T02:38:44Z)
• Proper Latent Decomposition [4.266376725904727]
  We compute a reduced set of intrinsic coordinates (latent space) to accurately describe a flow with fewer degrees of freedom than the numerical discretization.<n>With this proposed numerical framework, we propose an algorithm to perform PLD on the manifold.<n>This work opens opportunities for analyzing autoencoders and latent spaces, nonlinear reduced-order modeling and scientific insights into the structure of high-dimensional data.
  arXiv  Detail & Related papers  (2024-12-01T12:19:08Z)
• Stochastic Reconstruction of Gappy Lagrangian Turbulent Signals by Conditional Diffusion Models [1.7810134788247751]
  We present a method for reconstructing missing spatial and velocity data along the trajectories of small objects passively advected by turbulent flows. Our approach makes use of conditional generative diffusion models, a recently proposed data-driven machine learning technique.
  arXiv  Detail & Related papers  (2024-10-31T14:26:10Z)
• Bayesian Circular Regression with von Mises Quasi-Processes [57.88921637944379]
  In this work we explore a family of expressive and interpretable distributions over circle-valued random functions.<n>For posterior inference, we introduce a new Stratonovich-like augmentation that lends itself to fast Gibbs sampling.<n>We present experiments applying this model to the prediction of wind directions and the percentage of the running gait cycle as a function of joint angles.
  arXiv  Detail & Related papers  (2024-06-19T01:57:21Z)
• Score-based generative models learn manifold-like structures with constrained mixing [2.843124313496295]
  How do score-based generative models learn the data distribution supported on a low-dimensional manifold? We investigate the score model of a trained SBM through its linear approximations and subspaces spanned by local feature vectors. We find that the learned vector field mixes samples by a non-conservative field within the manifold, although it denoises with normal projections as if there is an energy function in off-manifold directions.
  arXiv  Detail & Related papers  (2023-11-16T15:15:15Z)
• Geometric Neural Diffusion Processes [55.891428654434634]
  We extend the framework of diffusion models to incorporate a series of geometric priors in infinite-dimension modelling. We show that with these conditions, the generative functional model admits the same symmetry.
  arXiv  Detail & Related papers  (2023-07-11T16:51:38Z)
• A Geometric Perspective on Diffusion Models [57.27857591493788]
  We inspect the ODE-based sampling of a popular variance-exploding SDE. We establish a theoretical relationship between the optimal ODE-based sampling and the classic mean-shift (mode-seeking) algorithm.
  arXiv  Detail & Related papers  (2023-05-31T15:33:16Z)
• Short and Straight: Geodesics on Differentiable Manifolds [6.85316573653194]
  In this work, we first analyse existing methods for computing length-minimising geodesics. Second, we propose a model-based parameterisation for distance fields and geodesic flows on continuous manifold. Third, we develop a curvature-based training mechanism, sampling and scaling points in regions of the manifold exhibiting larger values of the Ricci scalar.
  arXiv  Detail & Related papers  (2023-05-24T15:09:41Z)
• Random Smoothing Regularization in Kernel Gradient Descent Learning [24.383121157277007]
  We present a framework for random smoothing regularization that can adaptively learn a wide range of ground truth functions belonging to the classical Sobolev spaces. Our estimator can adapt to the structural assumptions of the underlying data and avoid the curse of dimensionality.
  arXiv  Detail & Related papers  (2023-05-05T13:37:34Z)
• Score-based Diffusion Models in Function Space [137.70916238028306]
  Diffusion models have recently emerged as a powerful framework for generative modeling.<n>This work introduces a mathematically rigorous framework called Denoising Diffusion Operators (DDOs) for training diffusion models in function space.<n>We show that the corresponding discretized algorithm generates accurate samples at a fixed cost independent of the data resolution.
  arXiv  Detail & Related papers  (2023-02-14T23:50:53Z)
• Score-based Continuous-time Discrete Diffusion Models [102.65769839899315]
  We extend diffusion models to discrete variables by introducing a Markov jump process where the reverse process denoises via a continuous-time Markov chain. We show that an unbiased estimator can be obtained via simple matching the conditional marginal distributions. We demonstrate the effectiveness of the proposed method on a set of synthetic and real-world music and image benchmarks.
  arXiv  Detail & Related papers  (2022-11-30T05:33:29Z)
• Diffusion Models for Causal Discovery via Topological Ordering [20.875222263955045]
  emphTopological ordering approaches reduce the optimisation space of causal discovery by searching over a permutation rather than graph space. For ANMs, the emphHessian of the data log-likelihood can be used for finding leaf nodes in a causal graph, allowing its topological ordering. We introduce theory for updating the learned Hessian without re-training the neural network, and we show that computing with a subset of samples gives an accurate approximation of the ordering.
  arXiv  Detail & Related papers  (2022-10-12T13:36:29Z)
• Manifold Interpolating Optimal-Transport Flows for Trajectory Inference [64.94020639760026]
  We present a method called Manifold Interpolating Optimal-Transport Flow (MIOFlow) MIOFlow learns, continuous population dynamics from static snapshot samples taken at sporadic timepoints. We evaluate our method on simulated data with bifurcations and merges, as well as scRNA-seq data from embryoid body differentiation, and acute myeloid leukemia treatment.
  arXiv  Detail & Related papers  (2022-06-29T22:19:03Z)
• Trajectory Inference via Mean-field Langevin in Path Space [0.17205106391379024]
  Trajectory inference aims at recovering the dynamics of a population from snapshots of its temporal marginals. A min-entropy estimator relative to the Wiener measure in path space was introduced by Lavenant et al.
  arXiv  Detail & Related papers  (2022-05-14T23:13:00Z)
• Riemannian Score-Based Generative Modeling [56.20669989459281]
  We introduce score-based generative models (SGMs) demonstrating remarkable empirical performance. Current SGMs make the underlying assumption that the data is supported on a Euclidean manifold with flat geometry. This prevents the use of these models for applications in robotics, geoscience or protein modeling.
  arXiv  Detail & Related papers  (2022-02-06T11:57:39Z)
• Inverting brain grey matter models with likelihood-free inference: a tool for trustable cytoarchitecture measurements [62.997667081978825]
  characterisation of the brain grey matter cytoarchitecture with quantitative sensitivity to soma density and volume remains an unsolved challenge in dMRI. We propose a new forward model, specifically a new system of equations, requiring a few relatively sparse b-shells. We then apply modern tools from Bayesian analysis known as likelihood-free inference (LFI) to invert our proposed model.
  arXiv  Detail & Related papers  (2021-11-15T09:08:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.