Latent SDEs on Homogeneous Spaces

• URL: http://arxiv.org/abs/2306.16248v3
• Date: Wed, 21 Feb 2024 14:11:19 GMT
• Title: Latent SDEs on Homogeneous Spaces
• Authors: Sebastian Zeng, Florian Graf, Roland Kwitt
• Abstract summary: We consider the problem of variational Bayesian inference in a latent variable model where a (possibly complex) observed geometric process is governed by the solution of a latent differential equation (SDE) Experiments demonstrate that a latent SDE of the proposed type can be learned efficiently by means of an existing one-step Euler-Maruyama scheme.
• Score: 9.361372513858043
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We consider the problem of variational Bayesian inference in a latent variable model where a (possibly complex) observed stochastic process is governed by the solution of a latent stochastic differential equation (SDE). Motivated by the challenges that arise when trying to learn an (almost arbitrary) latent neural SDE from data, such as efficient gradient computation, we take a step back and study a specific subclass instead. In our case, the SDE evolves on a homogeneous latent space and is induced by stochastic dynamics of the corresponding (matrix) Lie group. In learning problems, SDEs on the unit n-sphere are arguably the most relevant incarnation of this setup. Notably, for variational inference, the sphere not only facilitates using a truly uninformative prior, but we also obtain a particularly simple and intuitive expression for the Kullback-Leibler divergence between the approximate posterior and prior process in the evidence lower bound. Experiments demonstrate that a latent SDE of the proposed type can be learned efficiently by means of an existing one-step geometric Euler-Maruyama scheme. Despite restricting ourselves to a less rich class of SDEs, we achieve competitive or even state-of-the-art results on various time series interpolation/classification problems.

Related papers

• Identifying Drift, Diffusion, and Causal Structure from Temporal Snapshots [10.018568337210876]
  We present the first comprehensive approach for jointly estimating the drift and diffusion of an SDE from its temporal marginals. We show that each of these steps areAlterally optimal with respect to the Kullback-Leibler datasets.
  arXiv  Detail & Related papers  (2024-10-30T06:28:21Z)
• Total Uncertainty Quantification in Inverse PDE Solutions Obtained with Reduced-Order Deep Learning Surrogate Models [50.90868087591973]
  We propose an approximate Bayesian method for quantifying the total uncertainty in inverse PDE solutions obtained with machine learning surrogate models. We test the proposed framework by comparing it with the iterative ensemble smoother and deep ensembling methods for a non-linear diffusion equation.
  arXiv  Detail & Related papers  (2024-08-20T19:06:02Z)
• AdjointDEIS: Efficient Gradients for Diffusion Models [2.0795007613453445]
  We show that continuous adjoint equations for diffusion SDEs actually simplify to a simple ODE. We also demonstrate the effectiveness of AdjointDEIS for guided generation with an adversarial attack in the form of the face morphing problem.
  arXiv  Detail & Related papers  (2024-05-23T19:51:33Z)
• Deep Equilibrium Based Neural Operators for Steady-State PDEs [100.88355782126098]
  We study the benefits of weight-tied neural network architectures for steady-state PDEs. We propose FNO-DEQ, a deep equilibrium variant of the FNO architecture that directly solves for the solution of a steady-state PDE.
  arXiv  Detail & Related papers  (2023-11-30T22:34:57Z)
• Gaussian Mixture Solvers for Diffusion Models [84.83349474361204]
  We introduce a novel class of SDE-based solvers called GMS for diffusion models. Our solver outperforms numerous SDE-based solvers in terms of sample quality in image generation and stroke-based synthesis.
  arXiv  Detail & Related papers  (2023-11-02T02:05:38Z)
• Learning differentiable solvers for systems with hard constraints [48.54197776363251]
  We introduce a practical method to enforce partial differential equation (PDE) constraints for functions defined by neural networks (NNs) We develop a differentiable PDE-constrained layer that can be incorporated into any NN architecture. Our results show that incorporating hard constraints directly into the NN architecture achieves much lower test error when compared to training on an unconstrained objective.
  arXiv  Detail & Related papers  (2022-07-18T15:11:43Z)
• Lie Point Symmetry Data Augmentation for Neural PDE Solvers [69.72427135610106]
  We present a method, which can partially alleviate this problem, by improving neural PDE solver sample complexity. In the context of PDEs, it turns out that we are able to quantitatively derive an exhaustive list of data transformations. We show how it can easily be deployed to improve neural PDE solver sample complexity by an order of magnitude.
  arXiv  Detail & Related papers  (2022-02-15T18:43:17Z)
• Learning stochastic dynamical systems with neural networks mimicking the Euler-Maruyama scheme [14.436723124352817]
  We propose a data driven approach where parameters of the SDE are represented by a neural network with a built-in SDE integration scheme. The algorithm is applied to the geometric brownian motion and a version of the Lorenz-63 model.
  arXiv  Detail & Related papers  (2021-05-18T11:41:34Z)
• Neural SDEs as Infinite-Dimensional GANs [18.07683058213448]
  We show that the current classical approach to fitting SDEs may be approached as a special case of (Wasserstein) GANs. We obtain Neural SDEs as (in modern machine learning parlance) continuous-time generative time series models.
  arXiv  Detail & Related papers  (2021-02-06T19:59:15Z)
• Identifying Latent Stochastic Differential Equations [29.103393300261587]
  We present a method for learning latent differential equations (SDEs) from high-dimensional time series data. The proposed method learns the mapping from ambient to latent space, and the underlying SDE coefficients, through a self-supervised learning approach. We validate the method through several simulated video processing tasks, where the underlying SDE is known, and through real world datasets.
  arXiv  Detail & Related papers  (2020-07-12T19:46:31Z)
• Stochastic Normalizing Flows [52.92110730286403]
  We introduce normalizing flows for maximum likelihood estimation and variational inference (VI) using differential equations (SDEs) Using the theory of rough paths, the underlying Brownian motion is treated as a latent variable and approximated, enabling efficient training of neural SDEs. These SDEs can be used for constructing efficient chains to sample from the underlying distribution of a given dataset.
  arXiv  Detail & Related papers  (2020-02-21T20:47:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.