Deep Gaussian Markov Random Fields

• URL: http://arxiv.org/abs/2002.07467v2
• Date: Mon, 10 Aug 2020 15:19:04 GMT
• Title: Deep Gaussian Markov Random Fields
• Authors: Per Sid\'en and Fredrik Lindsten
• Abstract summary: We establish a formal connection between GMRFs and convolutional neural networks (CNNs) Common GMRFs are special cases of a generative model where the inverse mapping from data to latent variables is given by a 1-layer linear CNN. We describe how well-established tools, such as autodiff and variational inference, can be used for simple and efficient inference and learning of the deep GMRF.
• Score: 17.31058900857327
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Gaussian Markov random fields (GMRFs) are probabilistic graphical models widely used in spatial statistics and related fields to model dependencies over spatial structures. We establish a formal connection between GMRFs and convolutional neural networks (CNNs). Common GMRFs are special cases of a generative model where the inverse mapping from data to latent variables is given by a 1-layer linear CNN. This connection allows us to generalize GMRFs to multi-layer CNN architectures, effectively increasing the order of the corresponding GMRF in a way which has favorable computational scaling. We describe how well-established tools, such as autodiff and variational inference, can be used for simple and efficient inference and learning of the deep GMRF. We demonstrate the flexibility of the proposed model and show that it outperforms the state-of-the-art on a dataset of satellite temperatures, in terms of prediction and predictive uncertainty.

Related papers

• Streaming Gaussian Dirichlet Random Fields for Spatial Predictions of High Dimensional Categorical Observations [0.0]
  We present a novel approach to a stream oftemporally distributed, sparse, high-dimensional categorical observations. The proposed approach efficiently learns global local patterns in Streaming data. We demonstrate the ability of a network approach to make more accurate predictions.
  arXiv  Detail & Related papers  (2024-02-23T14:52:05Z)
• Random forests for binary geospatial data [0.0]
  We propose RF-GP, using Random Forests for estimating the non-linear covariate effect and Gaussian Processes for modeling the spatial random effects. RF-GP outperforms existing RF methods for estimation and prediction in both simulated and real-world data. We establish consistency of RF-GP for a general class of $beta$-mixing binary processes that includes common choices like spatial Mat'ern GP and autoregressive processes.
  arXiv  Detail & Related papers  (2023-02-27T14:34:33Z)
• Functional Neural Networks: Shift invariant models for functional data with applications to EEG classification [0.0]
  We introduce a new class of neural networks that are shift invariant and preserve smoothness of the data: functional neural networks (FNNs) For this, we use methods from functional data analysis (FDA) to extend multi-layer perceptrons and convolutional neural networks to functional data. We show that the models outperform a benchmark model from FDA in terms of accuracy and successfully use FNNs to classify electroencephalography (EEG) data.
  arXiv  Detail & Related papers  (2023-01-14T09:41:21Z)
• Learning from aggregated data with a maximum entropy model [73.63512438583375]
  We show how a new model, similar to a logistic regression, may be learned from aggregated data only by approximating the unobserved feature distribution with a maximum entropy hypothesis. We present empirical evidence on several public datasets that the model learned this way can achieve performances comparable to those of a logistic model trained with the full unaggregated data.
  arXiv  Detail & Related papers  (2022-10-05T09:17:27Z)
• Learnable Filters for Geometric Scattering Modules [64.03877398967282]
  We propose a new graph neural network (GNN) module based on relaxations of recently proposed geometric scattering transforms. Our learnable geometric scattering (LEGS) module enables adaptive tuning of the wavelets to encourage band-pass features to emerge in learned representations.
  arXiv  Detail & Related papers  (2022-08-15T22:30:07Z)
• Scalable Deep Gaussian Markov Random Fields for General Graphs [14.653008985229615]
  We propose a flexible GMRF model for general graphs built on the multi-layer structure of Deep GMRFs. For a Gaussian likelihood, close to exact Bayesian inference is available for the latent field. The usefulness of the proposed model is verified by experiments on a number of synthetic and real world datasets.
  arXiv  Detail & Related papers  (2022-06-10T12:12:41Z)
• Crime Prediction with Graph Neural Networks and Multivariate Normal Distributions [18.640610803366876]
  We tackle the sparsity problem in high resolution by leveraging the flexible structure of graph convolutional networks (GCNs) We build our model with Graph Convolutional Gated Recurrent Units (Graph-ConvGRU) to learn spatial, temporal, and categorical relations. We show that our model is not only generative but also precise.
  arXiv  Detail & Related papers  (2021-11-29T17:37:01Z)
• Rank-R FNN: A Tensor-Based Learning Model for High-Order Data Classification [69.26747803963907]
  Rank-R Feedforward Neural Network (FNN) is a tensor-based nonlinear learning model that imposes Canonical/Polyadic decomposition on its parameters. First, it handles inputs as multilinear arrays, bypassing the need for vectorization, and can thus fully exploit the structural information along every data dimension. We establish the universal approximation and learnability properties of Rank-R FNN, and we validate its performance on real-world hyperspectral datasets.
  arXiv  Detail & Related papers  (2021-04-11T16:37:32Z)
• GANs with Conditional Independence Graphs: On Subadditivity of Probability Divergences [70.30467057209405]
  Generative Adversarial Networks (GANs) are modern methods to learn the underlying distribution of a data set. GANs are designed in a model-free fashion where no additional information about the underlying distribution is available. We propose a principled design of a model-based GAN that uses a set of simple discriminators on the neighborhoods of the Bayes-net/MRF.
  arXiv  Detail & Related papers  (2020-03-02T04:31:22Z)
• Block-Approximated Exponential Random Graphs [77.4792558024487]
  An important challenge in the field of exponential random graphs (ERGs) is the fitting of non-trivial ERGs on large graphs. We propose an approximative framework to such non-trivial ERGs that result in dyadic independence (i.e., edge independent) distributions. Our methods are scalable to sparse graphs consisting of millions of nodes.
  arXiv  Detail & Related papers  (2020-02-14T11:42:16Z)
• Model Fusion via Optimal Transport [64.13185244219353]
  We present a layer-wise model fusion algorithm for neural networks. We show that this can successfully yield "one-shot" knowledge transfer between neural networks trained on heterogeneous non-i.i.d. data.
  arXiv  Detail & Related papers  (2019-10-12T22:07:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.