Spectral Diffusion Processes

• URL: http://arxiv.org/abs/2209.14125v1
• Date: Wed, 28 Sep 2022 14:23:41 GMT
• Title: Spectral Diffusion Processes
• Authors: Angus Phillips, Thomas Seror, Michael Hutchinson, Valentin De Bortoli, Arnaud Doucet, Emile Mathieu
• Abstract summary: Score-based generative modelling (SGM) has proven to be a very effective method for modelling densities on finite-dimensional spaces. We represent functional data in spectral space to dissociate part of the processes from their space-time part.
• Score: 26.510979162244304
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Score-based generative modelling (SGM) has proven to be a very effective method for modelling densities on finite-dimensional spaces. In this work we propose to extend this methodology to learn generative models over functional spaces. To do so, we represent functional data in spectral space to dissociate the stochastic part of the processes from their space-time part. Using dimensionality reduction techniques we then sample from their stochastic component using finite dimensional SGM. We demonstrate our method's effectiveness for modelling various multimodal datasets.

Related papers

• Geometric Neural Diffusion Processes [55.891428654434634]
  We extend the framework of diffusion models to incorporate a series of geometric priors in infinite-dimension modelling. We show that with these conditions, the generative functional model admits the same symmetry.
  arXiv  Detail & Related papers  (2023-07-11T16:51:38Z)
• Functional Flow Matching [14.583771853250008]
  We propose a function-space generative model that generalizes the recently-introduced Flow Matching model. Our method does not rely on likelihoods or simulations, making it well-suited to the function space setting. We demonstrate through experiments on several real-world benchmarks that our proposed FFM method outperforms several recently proposed function-space generative models.
  arXiv  Detail & Related papers  (2023-05-26T19:07:47Z)
• Trans-Dimensional Generative Modeling via Jump Diffusion Models [46.183265841345644]
  We propose a new class of generative models that naturally handle data of varying dimensionality. We first define a dimension destroying forward noising process, before deriving the dimension creating time-reversed generative process. Simulating our learned approximation to the time-reversed generative process then provides an effective way of sampling data of varying dimensionality.
  arXiv  Detail & Related papers  (2023-05-25T17:15:00Z)
• Clustering based on Mixtures of Sparse Gaussian Processes [6.939768185086753]
  How to cluster data using their low dimensional embedded space is still a challenging problem in machine learning. In this article, we focus on proposing a joint formulation for both clustering and dimensionality reduction. Our algorithm is based on a mixture of sparse Gaussian processes, which is called Sparse Gaussian Process Mixture Clustering (SGP-MIC)
  arXiv  Detail & Related papers  (2023-03-23T20:44:36Z)
• Score-based Diffusion Models in Function Space [140.792362459734]
  Diffusion models have recently emerged as a powerful framework for generative modeling. We introduce a mathematically rigorous framework called Denoising Diffusion Operators (DDOs) for training diffusion models in function space. We show that the corresponding discretized algorithm generates accurate samples at a fixed cost independent of the data resolution.
  arXiv  Detail & Related papers  (2023-02-14T23:50:53Z)
• Diffusion Generative Models in Infinite Dimensions [10.15736468214228]
  We generalize diffusion generative models to operate directly in function space. A significant benefit of our function space point of view is that it allows us to explicitly specify the space of functions we are working in. Our approach allows us to perform both unconditional and conditional generation of function-valued data.
  arXiv  Detail & Related papers  (2022-12-01T21:54:19Z)
• Counting Phases and Faces Using Bayesian Thermodynamic Integration [77.34726150561087]
  We introduce a new approach to reconstruction of the thermodynamic functions and phase boundaries in two-parametric statistical mechanics systems. We use the proposed approach to accurately reconstruct the partition functions and phase diagrams of the Ising model and the exactly solvable non-equilibrium TASEP.
  arXiv  Detail & Related papers  (2022-05-18T17:11:23Z)
• Riemannian Score-Based Generative Modeling [56.20669989459281]
  We introduce score-based generative models (SGMs) demonstrating remarkable empirical performance. Current SGMs make the underlying assumption that the data is supported on a Euclidean manifold with flat geometry. This prevents the use of these models for applications in robotics, geoscience or protein modeling.
  arXiv  Detail & Related papers  (2022-02-06T11:57:39Z)
• On Contrastive Representations of Stochastic Processes [53.21653429290478]
  Learning representations of processes is an emerging problem in machine learning. We show that our methods are effective for learning representations of periodic functions, 3D objects and dynamical processes.
  arXiv  Detail & Related papers  (2021-06-18T11:00:24Z)
• Learning High-Dimensional Distributions with Latent Neural Fokker-Planck Kernels [67.81799703916563]
  We introduce new techniques to formulate the problem as solving Fokker-Planck equation in a lower-dimensional latent space. Our proposed model consists of latent-distribution morphing, a generator and a parameterized Fokker-Planck kernel function.
  arXiv  Detail & Related papers  (2021-05-10T17:42:01Z)
• Generative Archimedean Copulas [27.705956325584026]
  We propose a new generative modeling technique for learning multidimensional cumulative distribution functions (CDFs) in the form of copulas. We consider certain classes of copulas known as Archimedean and hierarchical Archimedean copulas, popular for their parsimonious representation and ability to model different tail dependencies.
  arXiv  Detail & Related papers  (2021-02-22T20:45:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.