A prescriptive theory for brain-like inference

• URL: http://arxiv.org/abs/2410.19315v1
• Date: Fri, 25 Oct 2024 06:00:18 GMT
• Title: A prescriptive theory for brain-like inference
• Authors: Hadi Vafaii, Dekel Galor, Jacob L. Yates,
• Abstract summary: We show that maximizing the Evidence Lower Bound (ELBO) leads to a spiking neural network that performs Bayesian posterior inference. The resulting model, the iterative Poisson VAE, has a closer connection to biological neurons than previous brain-inspired predictive models. These findings suggest that optimizing ELBO, combined with Poisson assumptions, provides a solid foundation for developing prescriptive theories in NeuroAI.
• Score: 0.0
• License:
• Abstract: The Evidence Lower Bound (ELBO) is a widely used objective for training deep generative models, such as Variational Autoencoders (VAEs). In the neuroscience literature, an identical objective is known as the variational free energy, hinting at a potential unified framework for brain function and machine learning. Despite its utility in interpreting generative models, including diffusion models, ELBO maximization is often seen as too broad to offer prescriptive guidance for specific architectures in neuroscience or machine learning. In this work, we show that maximizing ELBO under Poisson assumptions for general sequence data leads to a spiking neural network that performs Bayesian posterior inference through its membrane potential dynamics. The resulting model, the iterative Poisson VAE (iP-VAE), has a closer connection to biological neurons than previous brain-inspired predictive coding models based on Gaussian assumptions. Compared to amortized and iterative VAEs, iP-VAElearns sparser representations and exhibits superior generalization to out-of-distribution samples. These findings suggest that optimizing ELBO, combined with Poisson assumptions, provides a solid foundation for developing prescriptive theories in NeuroAI.

Related papers

• A Non-negative VAE:the Generalized Gamma Belief Network [49.970917207211556]
  The gamma belief network (GBN) has demonstrated its potential for uncovering multi-layer interpretable latent representations in text data. We introduce the generalized gamma belief network (Generalized GBN) in this paper, which extends the original linear generative model to a more expressive non-linear generative model. We also propose an upward-downward Weibull inference network to approximate the posterior distribution of the latent variables.
  arXiv  Detail & Related papers  (2024-08-06T18:18:37Z)
• A Neural Network Implementation for Free Energy Principle [3.7937771805690392]
  The free energy principle (FEP) has been widely applied to account for various problems in fields such as cognitive science, neuroscience, social interaction, and hermeneutics. This paper gives a preliminary attempt at bridging FEP and machine learning, via a classical neural network model, the Helmholtz machine. Although the Helmholtz machine is not temporal, it gives an ideal parallel to the vanilla FEP and the hierarchical model of the brain, under which the active inference and predictive coding could be formulated coherently.
  arXiv  Detail & Related papers  (2023-06-11T22:14:21Z)
• Neural Frailty Machine: Beyond proportional hazard assumption in neural survival regressions [30.018173329118184]
  We present neural frailty machine (NFM), a powerful and flexible neural modeling framework for survival regressions. Two concrete models are derived under the framework that extends neural proportional hazard models and non hazard regression models. We conduct experimental evaluations over $6$ benchmark datasets of different scales, showing that the proposed NFM models outperform state-of-the-art survival models in terms of predictive performance.
  arXiv  Detail & Related papers  (2023-03-18T08:15:15Z)
• On the Generalization and Adaption Performance of Causal Models [99.64022680811281]
  Differentiable causal discovery has proposed to factorize the data generating process into a set of modules. We study the generalization and adaption performance of such modular neural causal models. Our analysis shows that the modular neural causal models outperform other models on both zero and few-shot adaptation in low data regimes.
  arXiv  Detail & Related papers  (2022-06-09T17:12:32Z)
• EINNs: Epidemiologically-Informed Neural Networks [75.34199997857341]
  We introduce a new class of physics-informed neural networks-EINN-crafted for epidemic forecasting. We investigate how to leverage both the theoretical flexibility provided by mechanistic models as well as the data-driven expressability afforded by AI models.
  arXiv  Detail & Related papers  (2022-02-21T18:59:03Z)
• Gone Fishing: Neural Active Learning with Fisher Embeddings [55.08537975896764]
  There is an increasing need for active learning algorithms that are compatible with deep neural networks. This article introduces BAIT, a practical representation of tractable, and high-performing active learning algorithm for neural networks.
  arXiv  Detail & Related papers  (2021-06-17T17:26:31Z)
• Cauchy-Schwarz Regularized Autoencoder [68.80569889599434]
  Variational autoencoders (VAE) are a powerful and widely-used class of generative models. We introduce a new constrained objective based on the Cauchy-Schwarz divergence, which can be computed analytically for GMMs. Our objective improves upon variational auto-encoding models in density estimation, unsupervised clustering, semi-supervised learning, and face analysis.
  arXiv  Detail & Related papers  (2021-01-06T17:36:26Z)
• The Neural Coding Framework for Learning Generative Models [91.0357317238509]
  We propose a novel neural generative model inspired by the theory of predictive processing in the brain. In a similar way, artificial neurons in our generative model predict what neighboring neurons will do, and adjust their parameters based on how well the predictions matched reality.
  arXiv  Detail & Related papers  (2020-12-07T01:20:38Z)
• Iterative VAE as a predictive brain model for out-of-distribution generalization [7.006301658267125]
  We show that iVAEs generalize to distributional shifts significantly better than both PCNs and VAEs. We propose a novel measure of recognizability for individual samples which can be tested against human psychophysical data.
  arXiv  Detail & Related papers  (2020-12-01T15:02:38Z)
• Improved Protein-ligand Binding Affinity Prediction with Structure-Based Deep Fusion Inference [3.761791311908692]
  Predicting accurate protein-ligand binding affinity is important in drug discovery. Recent advances in the deep convolutional and graph neural network based approaches, the model performance depends on the input data representation. We present fusion models to benefit from different feature representations of two neural network models to improve the binding affinity prediction.
  arXiv  Detail & Related papers  (2020-05-17T22:26:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.