Bayesian Learning via Neural Schr\"odinger-F\"ollmer Flows

• URL: http://arxiv.org/abs/2111.10510v2
• Date: Tue, 23 Nov 2021 18:14:56 GMT
• Title: Bayesian Learning via Neural Schr\"odinger-F\"ollmer Flows
• Authors: Francisco Vargas, Andrius Ovsianas, David Fernandes, Mark Girolami, Neil Lawrence, Nikolas N\"usken
• Abstract summary: We advocate control as a finite time alternative to popular steady-state methods such as gradient Langevin dynamics (SGLD) We discuss and adapt the existing theoretical guarantees of this framework and establish connections to already existing VI routines in SDE-based models.
• Score: 3.07869141026886
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work we explore a new framework for approximate Bayesian inference in large datasets based on stochastic control. We advocate stochastic control as a finite time alternative to popular steady-state methods such as stochastic gradient Langevin dynamics (SGLD). Furthermore, we discuss and adapt the existing theoretical guarantees of this framework and establish connections to already existing VI routines in SDE-based models.

Related papers

• Trajectory Flow Matching with Applications to Clinical Time Series Modeling [77.58277281319253]
  Trajectory Flow Matching (TFM) trains a Neural SDE in a simulation-free manner, bypassing backpropagation through the dynamics. We demonstrate improved performance on three clinical time series datasets in terms of absolute performance and uncertainty prediction.
  arXiv  Detail & Related papers  (2024-10-28T15:54:50Z)
• Partially Observed Trajectory Inference using Optimal Transport and a Dynamics Prior [2.7255073299359154]
  Trajectory inference seeks to recover the temporal dynamics of a population from snapshots of its temporal marginals. We introduce PO-MFL to solve this latent trajectory inference problem. We leverage the MFL framework of arXiv:2205.07146, yielding an algorithm based on entropic OT between dynamics-adjusted adjacent time marginals.
  arXiv  Detail & Related papers  (2024-06-11T17:21:15Z)
• Variational Inference for SDEs Driven by Fractional Noise [16.434973057669676]
  We present a novel variational framework for performing inference in (neural) differential equations (SDEs) driven by Markov-approximate fractional Brownian motion (fBM) We propose the use of neural networks to learn the drift, diffusion and control terms within our variational posterior leading to the variational training of neural-SDEs.
  arXiv  Detail & Related papers  (2023-10-19T17:59:21Z)
• Free-Form Variational Inference for Gaussian Process State-Space Models [21.644570034208506]
  We propose a new method for inference in Bayesian GPSSMs. Our method is based on freeform variational inference via inducing Hamiltonian Monte Carlo. We show that our approach can learn transition dynamics and latent states more accurately than competing methods.
  arXiv  Detail & Related papers  (2023-02-20T11:34:16Z)
• Guaranteed Conservation of Momentum for Learning Particle-based Fluid Dynamics [96.9177297872723]
  We present a novel method for guaranteeing linear momentum in learned physics simulations. We enforce conservation of momentum with a hard constraint, which we realize via antisymmetrical continuous convolutional layers. In combination, the proposed method allows us to increase the physical accuracy of the learned simulator substantially.
  arXiv  Detail & Related papers  (2022-10-12T09:12:59Z)
• Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equations [37.02511585732081]
  We perform scalable approximate inference in a recently-proposed family of continuous-depth neural networks. We demonstrate gradient-based variational inference, producing arbitrarily-flexible approximate posteriors. This approach further inherits the memory-efficient training and tunable precision of neural ODEs.
  arXiv  Detail & Related papers  (2021-02-12T14:48:58Z)
• Training Generative Adversarial Networks by Solving Ordinary Differential Equations [54.23691425062034]
  We study the continuous-time dynamics induced by GAN training. From this perspective, we hypothesise that instabilities in training GANs arise from the integration error. We experimentally verify that well-known ODE solvers (such as Runge-Kutta) can stabilise training.
  arXiv  Detail & Related papers  (2020-10-28T15:23:49Z)
• Momentum via Primal Averaging: Theoretical Insights and Learning Rate Schedules for Non-Convex Optimization [10.660480034605241]
  Momentum methods are now used pervasively within the machine learning community for non-training models such as deep neural networks. In this work we develop a Lyapunov analysis of SGD with momentum, by utilizing the SGD equivalent rewriting of the primal SGD method known as the SGDSPA) form.
  arXiv  Detail & Related papers  (2020-10-01T13:46:32Z)
• A Kernel-Based Approach to Non-Stationary Reinforcement Learning in Metric Spaces [53.47210316424326]
  KeRNS is an algorithm for episodic reinforcement learning in non-stationary Markov Decision Processes. We prove a regret bound that scales with the covering dimension of the state-action space and the total variation of the MDP with time.
  arXiv  Detail & Related papers  (2020-07-09T21:37:13Z)
• Systematic large flavor fTWA approach to interaction quenches in the Hubbard model [55.2480439325792]
  We study the nonequilibrium dynamics after an interaction quench in the two-dimensional Hubbard model using the recently introduced fermionic truncated Wigner approximation (fTWA) We show that fTWA is exact at least up to and including the prethermalization dynamics.
  arXiv  Detail & Related papers  (2020-07-09T20:59:49Z)
• Stochastic-Sign SGD for Federated Learning with Theoretical Guarantees [49.91477656517431]
  Quantization-based solvers have been widely adopted in Federated Learning (FL) No existing methods enjoy all the aforementioned properties. We propose an intuitively-simple yet theoretically-simple method based on SIGNSGD to bridge the gap.
  arXiv  Detail & Related papers  (2020-02-25T15:12:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.