Predictive Coding: a Theoretical and Experimental Review
- URL: http://arxiv.org/abs/2107.12979v1
- Date: Tue, 27 Jul 2021 17:44:21 GMT
- Title: Predictive Coding: a Theoretical and Experimental Review
- Authors: Beren Millidge, Anil Seth, Christopher L Buckley
- Abstract summary: We provide a comprehensive review of the core mathematical structure and logic of predictive coding.
We also review a wide range of classic and recent work within the framework, ranging from the neurobiologically realistic microcircuits that could implement predictive coding.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Predictive coding offers a potentially unifying account of cortical function
-- postulating that the core function of the brain is to minimize prediction
errors with respect to a generative model of the world. The theory is closely
related to the Bayesian brain framework and, over the last two decades, has
gained substantial influence in the fields of theoretical and cognitive
neuroscience. A large body of research has arisen based on both empirically
testing improved and extended theoretical and mathematical models of predictive
coding, as well as in evaluating their potential biological plausibility for
implementation in the brain and the concrete neurophysiological and
psychological predictions made by the theory. Despite this enduring popularity,
however, no comprehensive review of predictive coding theory, and especially of
recent developments in this field, exists. Here, we provide a comprehensive
review both of the core mathematical structure and logic of predictive coding,
thus complementing recent tutorials in the literature. We also review a wide
range of classic and recent work within the framework, ranging from the
neurobiologically realistic microcircuits that could implement predictive
coding, to the close relationship between predictive coding and the widely-used
backpropagation of error algorithm, as well as surveying the close
relationships between predictive coding and modern machine learning techniques.
Related papers
- CogDPM: Diffusion Probabilistic Models via Cognitive Predictive Coding [62.075029712357]
This work introduces the Cognitive Diffusion Probabilistic Models (CogDPM)
CogDPM features a precision estimation method based on the hierarchical sampling capabilities of diffusion models and weight the guidance with precision weights estimated by the inherent property of diffusion models.
We apply CogDPM to real-world prediction tasks using the United Kindom precipitation and surface wind datasets.
arXiv Detail & Related papers (2024-05-03T15:54:50Z) - Predictive Coding and Stochastic Resonance: Towards a Unified Theory of
Auditory (Phantom) Perception [6.416574036611064]
To gain a mechanistic understanding of brain function, hypothesis driven experiments should be accompanied by biologically plausible computational models.
With a special focus on tinnitus, we review recent work at the intersection of artificial intelligence, psychology, and neuroscience.
We conclude that two fundamental processing principles - being ubiquitous in the brain - best fit to a vast number of experimental results.
arXiv Detail & Related papers (2022-04-07T10:47:58Z) - Hybrid Predictive Coding: Inferring, Fast and Slow [62.997667081978825]
We propose a hybrid predictive coding network that combines both iterative and amortized inference in a principled manner.
We demonstrate that our model is inherently sensitive to its uncertainty and adaptively balances balances to obtain accurate beliefs using minimum computational expense.
arXiv Detail & Related papers (2022-04-05T12:52:45Z) - Long-range and hierarchical language predictions in brains and
algorithms [82.81964713263483]
We show that while deep language algorithms are optimized to predict adjacent words, the human brain would be tuned to make long-range and hierarchical predictions.
This study strengthens predictive coding theory and suggests a critical role of long-range and hierarchical predictions in natural language processing.
arXiv Detail & Related papers (2021-11-28T20:26:07Z) - The Neural Coding Framework for Learning Generative Models [91.0357317238509]
We propose a novel neural generative model inspired by the theory of predictive processing in the brain.
In a similar way, artificial neurons in our generative model predict what neighboring neurons will do, and adjust their parameters based on how well the predictions matched reality.
arXiv Detail & Related papers (2020-12-07T01:20:38Z) - Variational Bayes Neural Network: Posterior Consistency, Classification
Accuracy and Computational Challenges [0.3867363075280544]
This paper develops a variational Bayesian neural network estimation methodology and related statistical theory.
The development is motivated by an important biomedical engineering application, namely building predictive tools for the transition from mild cognitive impairment to Alzheimer's disease.
arXiv Detail & Related papers (2020-11-19T00:11:27Z) - Relaxing the Constraints on Predictive Coding Models [62.997667081978825]
Predictive coding is an influential theory of cortical function which posits that the principal computation the brain performs is the minimization of prediction errors.
Standard implementations of the algorithm still involve potentially neurally implausible features such as identical forward and backward weights, backward nonlinear derivatives, and 1-1 error unit connectivity.
In this paper, we show that these features are not integral to the algorithm and can be removed either directly or through learning additional sets of parameters with Hebbian update rules without noticeable harm to learning performance.
arXiv Detail & Related papers (2020-10-02T15:21:37Z) - Neuro-symbolic Neurodegenerative Disease Modeling as Probabilistic
Programmed Deep Kernels [93.58854458951431]
We present a probabilistic programmed deep kernel learning approach to personalized, predictive modeling of neurodegenerative diseases.
Our analysis considers a spectrum of neural and symbolic machine learning approaches.
We run evaluations on the problem of Alzheimer's disease prediction, yielding results that surpass deep learning.
arXiv Detail & Related papers (2020-09-16T15:16:03Z) - Predictive coding in balanced neural networks with noise, chaos and
delays [24.76770648963407]
We introduce an analytically tractable model of balanced predictive coding, in which the degree of balance and the degree of weight disorder can be dissociated.
Our work provides and solves a general theoretical framework for dissecting the differential contributions neural noise, synaptic disorder, chaos, synaptic delays, and balance to the fidelity of predictive neural codes.
arXiv Detail & Related papers (2020-06-25T05:03:27Z) - Rationalizing Medical Relation Prediction from Corpus-level Statistics [27.5727760575915]
We present a new interpretable framework inspired by existing theories on how human memory works, e.g., theories of recall and recognition.
We conduct experiments on a real-world public clinical dataset and show that our framework can not only achieve competitive predictive performance against a comprehensive list of neural baseline models, but also present rationales to justify its prediction.
arXiv Detail & Related papers (2020-05-02T17:39:40Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.