Equivariant Learning of Stochastic Fields: Gaussian Processes and Steerable Conditional Neural Processes

• URL: http://arxiv.org/abs/2011.12916v3
• Date: Sat, 17 Jul 2021 13:09:15 GMT
• Title: Equivariant Learning of Stochastic Fields: Gaussian Processes and Steerable Conditional Neural Processes
• Authors: Peter Holderrieth, Michael Hutchinson, Yee Whye Teh
• Abstract summary: We study the problem of learning fields, i.e. processes whose samples are fields like those occurring in physics and engineering. We introduce Steerable Conditional Neural Processes (SteerCNPs), a new, fully equivariant member of the Neural Process family. In experiments with Gaussian process vector fields, images, and real-world weather data, we observe that SteerCNPs significantly improve the performance of previous models.
• Score: 44.51932024971217
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Motivated by objects such as electric fields or fluid streams, we study the problem of learning stochastic fields, i.e. stochastic processes whose samples are fields like those occurring in physics and engineering. Considering general transformations such as rotations and reflections, we show that spatial invariance of stochastic fields requires an inference model to be equivariant. Leveraging recent advances from the equivariance literature, we study equivariance in two classes of models. Firstly, we fully characterise equivariant Gaussian processes. Secondly, we introduce Steerable Conditional Neural Processes (SteerCNPs), a new, fully equivariant member of the Neural Process family. In experiments with Gaussian process vector fields, images, and real-world weather data, we observe that SteerCNPs significantly improve the performance of previous models and equivariance leads to improvements in transfer learning tasks.

Related papers

• Approximately Equivariant Neural Processes [47.14384085714576]
  We consider the use of approximately equivariant architectures in neural processes. We demonstrate the effectiveness of our approach on a number of synthetic and real-world regression experiments.
  arXiv  Detail & Related papers  (2024-06-19T12:17:14Z)
• Practical Equivariances via Relational Conditional Neural Processes [20.192899181958264]
  Conditional Neural Processes (CNPs) are a class of Conditional meta models popular for combining efficiency of amortized quantification with reliable uncertainty. We propose Conditional Neural Processes (RCNPs) as an effective approach to incorporate equivariances into any neural process model. Our proposed method extends the applicability and impact of equivariant processes to higher dimensions.
  arXiv  Detail & Related papers  (2023-06-19T13:27:27Z)
• Variational Gaussian Process Diffusion Processes [17.716059928867345]
  Diffusion processes are a class of differential equations (SDEs) providing a rich family of expressive models. Probabilistic inference and learning under generative models with latent processes endowed with a non-linear diffusion process prior are intractable problems. We build upon work within variational inference, approximating the posterior process as a linear diffusion process, and point out pathologies in the approach.
  arXiv  Detail & Related papers  (2023-06-03T09:43:59Z)
• Equivariance Discovery by Learned Parameter-Sharing [153.41877129746223]
  We study how to discover interpretable equivariances from data. Specifically, we formulate this discovery process as an optimization problem over a model's parameter-sharing schemes. Also, we theoretically analyze the method for Gaussian data and provide a bound on the mean squared gap between the studied discovery scheme and the oracle scheme.
  arXiv  Detail & Related papers  (2022-04-07T17:59:19Z)
• Equivariant vector field network for many-body system modeling [65.22203086172019]
  Equivariant Vector Field Network (EVFN) is built on a novel equivariant basis and the associated scalarization and vectorization layers. We evaluate our method on predicting trajectories of simulated Newton mechanics systems with both full and partially observed data.
  arXiv  Detail & Related papers  (2021-10-26T14:26:25Z)
• Learning Equivariant Energy Based Models with Equivariant Stein Variational Gradient Descent [80.73580820014242]
  We focus on the problem of efficient sampling and learning of probability densities by incorporating symmetries in probabilistic models. We first introduce Equivariant Stein Variational Gradient Descent algorithm -- an equivariant sampling method based on Stein's identity for sampling from densities with symmetries. We propose new ways of improving and scaling up training of energy based models.
  arXiv  Detail & Related papers  (2021-06-15T01:35:17Z)
• Recyclable Gaussian Processes [0.0]
  We present a new framework for recycling independent variational approximations to Gaussian processes. The main contribution is the construction of variational ensembles given a dictionary of fitted Gaussian processes. Our framework allows for regression, classification and heterogeneous tasks.
  arXiv  Detail & Related papers  (2020-10-06T09:01:55Z)
• Multiplicative noise and heavy tails in stochastic optimization [62.993432503309485]
  empirical optimization is central to modern machine learning, but its role in its success is still unclear. We show that it commonly arises in parameters of discrete multiplicative noise due to variance. A detailed analysis is conducted in which we describe on key factors, including recent step size, and data, all exhibit similar results on state-of-the-art neural network models.
  arXiv  Detail & Related papers  (2020-06-11T09:58:01Z)
• Transport Gaussian Processes for Regression [0.22843885788439797]
  We propose a methodology to construct processes, which include GPs, warped GPs, Student-t processes and several others. Our approach is inspired by layers-based models, where each proposed layer changes a specific property over the generated process. We validate the proposed model through experiments with real-world data.
  arXiv  Detail & Related papers  (2020-01-30T17:44:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.