Sparse Implicit Processes for Approximate Inference

• URL: http://arxiv.org/abs/2110.07618v1
• Date: Thu, 14 Oct 2021 18:00:04 GMT
• Title: Sparse Implicit Processes for Approximate Inference
• Authors: Sim\'on Rodr\'iguez Santana, Bryan Zaldivar, Daniel Hern\'andez-Lobato
• Abstract summary: Implicit processes (IPs) are flexible priors that can describe models such as Bayesian neural networks, neural samplers and data generators. We propose a method for simultaneously adjusting the parameters of the prior IP and the approximate posterior IP. The result is a scalable method for approximate inference with IPs that can tune the prior IP parameters to the data, and that provides accurate non-Gaussian predictive distributions.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Implicit Processes (IPs) are flexible priors that can describe models such as Bayesian neural networks, neural samplers and data generators. IPs allow for approximate inference in function-space. This avoids some degenerate problems of parameter-space approximate inference due to the high number of parameters and strong dependencies. For this, an extra IP is often used to approximate the posterior of the prior IP. However, simultaneously adjusting the parameters of the prior IP and the approximate posterior IP is a challenging task. Existing methods that can tune the prior IP result in a Gaussian predictive distribution, which fails to capture important data patterns. By contrast, methods producing flexible predictive distributions by using another IP to approximate the posterior process cannot fit the prior IP to the observed data. We propose here a method that can carry out both tasks. For this, we rely on an inducing-point representation of the prior IP, as often done in the context of sparse Gaussian processes. The result is a scalable method for approximate inference with IPs that can tune the prior IP parameters to the data, and that provides accurate non-Gaussian predictive distributions.

Related papers

• Unrolled denoising networks provably learn optimal Bayesian inference [54.79172096306631]
  We prove the first rigorous learning guarantees for neural networks based on unrolling approximate message passing (AMP) For compressed sensing, we prove that when trained on data drawn from a product prior, the layers of the network converge to the same denoisers used in Bayes AMP.
  arXiv  Detail & Related papers  (2024-09-19T17:56:16Z)
• A Meta-Learning Approach to Predicting Performance and Data Requirements [163.4412093478316]
  We propose an approach to estimate the number of samples required for a model to reach a target performance. We find that the power law, the de facto principle to estimate model performance, leads to large error when using a small dataset. We introduce a novel piecewise power law (PPL) that handles the two data differently.
  arXiv  Detail & Related papers  (2023-03-02T21:48:22Z)
• FaDIn: Fast Discretized Inference for Hawkes Processes with General Parametric Kernels [82.53569355337586]
  This work offers an efficient solution to temporal point processes inference using general parametric kernels with finite support. The method's effectiveness is evaluated by modeling the occurrence of stimuli-induced patterns from brain signals recorded with magnetoencephalography (MEG) Results show that the proposed approach leads to an improved estimation of pattern latency than the state-of-the-art.
  arXiv  Detail & Related papers  (2022-10-10T12:35:02Z)
• Correcting Model Bias with Sparse Implicit Processes [0.9187159782788579]
  We show that Sparse Implicit Processes (SIP) is capable of correcting model bias when the data generating mechanism differs strongly from the one implied by the model. We use synthetic datasets to show that SIP is capable of providing predictive distributions that reflect the data better than the exact predictions of the initial, but wrongly assumed model.
  arXiv  Detail & Related papers  (2022-07-21T18:00:01Z)
• Deep Variational Implicit Processes [0.0]
  Implicit processes (IPs) are a generalization of Gaussian processes (GPs) We propose a multi-layer generalization of IPs called the Deep Variational Implicit process (DVIP)
  arXiv  Detail & Related papers  (2022-06-14T10:04:41Z)
• Transformers Can Do Bayesian Inference [56.99390658880008]
  We present Prior-Data Fitted Networks (PFNs) PFNs leverage in-context learning in large-scale machine learning techniques to approximate a large set of posteriors. We demonstrate that PFNs can near-perfectly mimic Gaussian processes and also enable efficient Bayesian inference for intractable problems.
  arXiv  Detail & Related papers  (2021-12-20T13:07:39Z)
• Scaling Structured Inference with Randomization [64.18063627155128]
  We propose a family of dynamic programming (RDP) randomized for scaling structured models to tens of thousands of latent states. Our method is widely applicable to classical DP-based inference. It is also compatible with automatic differentiation so can be integrated with neural networks seamlessly.
  arXiv  Detail & Related papers  (2021-12-07T11:26:41Z)
• Mitigating Performance Saturation in Neural Marked Point Processes: Architectures and Loss Functions [50.674773358075015]
  We propose a simple graph-based network structure called GCHP, which utilizes only graph convolutional layers. We show that GCHP can significantly reduce training time and the likelihood ratio loss with interarrival time probability assumptions can greatly improve the model performance.
  arXiv  Detail & Related papers  (2021-07-07T16:59:14Z)
• Iterative Methods for Private Synthetic Data: Unifying Framework and New Methods [18.317488965846636]
  We study private synthetic data generation for query release. The goal is to construct a sanitized version of a sensitive dataset subject to differential privacy. Under this framework, we propose two new methods.
  arXiv  Detail & Related papers  (2021-06-14T04:19:35Z)
• A Graph Neural Network Approach for Scalable and Dynamic IP Similarity in Enterprise Networks [1.6516902135723865]
  Measuring similarity between IP addresses is an important task in the daily operations of any enterprise network. In this paper, we propose a novel approach for IP embedding using an adapted graph neural network (GNN) architecture.
  arXiv  Detail & Related papers  (2020-10-09T19:43:30Z)
• Scalable Plug-and-Play ADMM with Convergence Guarantees [24.957046830965822]
  We propose an incremental variant of the widely used. ADMM algorithm, making it scalable to large-scale datasets. We theoretically analyze the convergence algorithm under a set explicit assumptions.
  arXiv  Detail & Related papers  (2020-06-05T04:10:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.