A Mathematical Walkthrough and Discussion of the Free Energy Principle
- URL: http://arxiv.org/abs/2108.13343v1
- Date: Mon, 30 Aug 2021 16:11:49 GMT
- Title: A Mathematical Walkthrough and Discussion of the Free Energy Principle
- Authors: Beren Millidge, Anil Seth, Christopher L Buckley
- Abstract summary: The Free-Energy-Principle (FEP) is an influential and controversial theory which postulates a connection between the thermodynamics of self-organization and learning through variational inference.
FEP has been applied extensively in neuroscience, and is beginning to make inroads in machine learning by spurring the construction of novel and powerful algorithms by which action, perception, and learning can all be unified under a single objective.
Here, we aim to provide a mathematically detailed, yet intuitive walk-through of the formulation and central claims of the FEP while also providing a discussion of the assumptions necessary and potential limitations of the theory.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The Free-Energy-Principle (FEP) is an influential and controversial theory
which postulates a deep and powerful connection between the stochastic
thermodynamics of self-organization and learning through variational inference.
Specifically, it claims that any self-organizing system which can be
statistically separated from its environment, and which maintains itself at a
non-equilibrium steady state, can be construed as minimizing an
information-theoretic functional -- the variational free energy -- and thus
performing variational Bayesian inference to infer the hidden state of its
environment. This principle has also been applied extensively in neuroscience,
and is beginning to make inroads in machine learning by spurring the
construction of novel and powerful algorithms by which action, perception, and
learning can all be unified under a single objective. While its expansive and
often grandiose claims have spurred significant debates in both philosophy and
theoretical neuroscience, the mathematical depth and lack of accessible
introductions and tutorials for the core claims of the theory have often
precluded a deep understanding within the literature. Here, we aim to provide a
mathematically detailed, yet intuitive walk-through of the formulation and
central claims of the FEP while also providing a discussion of the assumptions
necessary and potential limitations of the theory. Additionally, since the FEP
is a still a living theory, subject to internal controversy, change, and
revision, we also present a detailed appendix highlighting and condensing
current perspectives as well as controversies about the nature, applicability,
and the mathematical assumptions and formalisms underlying the FEP.
Related papers
- A Theoretical Survey on Foundation Models [48.2313835471321]
This survey aims to review those interpretable methods that comply with the aforementioned principles and have been successfully applied to black-box foundation models.
The methods are deeply rooted in machine learning theory, covering the analysis of generalization performance, expressive capability, and dynamic behavior.
They provide a thorough interpretation of the entire workflow of FMs, ranging from the inference capability and training dynamics to their ethical implications.
arXiv Detail & Related papers (2024-10-15T09:48:03Z) - Relaxation of first-class constraints and the quantization of gauge theories: from "matter without matter" to the reappearance of time in quantum gravity [72.27323884094953]
We make a conceptual overview of an approach to the initial-value problem in canonical gauge theories.
We stress how the first-class phase-space constraints may be relaxed if we interpret them as fixing the values of new degrees of freedom.
arXiv Detail & Related papers (2024-02-19T19:00:02Z) - Acquiring and Modelling Abstract Commonsense Knowledge via Conceptualization [49.00409552570441]
We study the role of conceptualization in commonsense reasoning, and formulate a framework to replicate human conceptual induction.
We apply the framework to ATOMIC, a large-scale human-annotated CKG, aided by the taxonomy Probase.
arXiv Detail & Related papers (2022-06-03T12:24:49Z) - Fact-nets: towards a mathematical framework for relational quantum
mechanics [0.0]
The relational interpretation of quantum mechanics (RQM) has received a growing interest since its first formulation in 1996.
This paper proposes a radical reformulation of the mathematical framework of quantum mechanics which is relational from the start: fact-nets.
arXiv Detail & Related papers (2022-04-01T10:27:38Z) - Active Inference in Robotics and Artificial Agents: Survey and
Challenges [51.29077770446286]
We review the state-of-the-art theory and implementations of active inference for state-estimation, control, planning and learning.
We showcase relevant experiments that illustrate its potential in terms of adaptation, generalization and robustness.
arXiv Detail & Related papers (2021-12-03T12:10:26Z) - Quantum realism: axiomatization and quantification [77.34726150561087]
We build an axiomatization for quantum realism -- a notion of realism compatible with quantum theory.
We explicitly construct some classes of entropic quantifiers that are shown to satisfy almost all of the proposed axioms.
arXiv Detail & Related papers (2021-10-10T18:08:42Z) - Applications of the Free Energy Principle to Machine Learning and
Neuroscience [0.0]
We explore and apply methods inspired by the free energy principle to two important areas in machine learning and neuroscience.
We focus on predictive coding, a neurobiologically plausible process theory derived from the free energy principle.
Secondly, we study active inference, a neurobiologically grounded account of action through variational message passing.
Finally, we investigate biologically plausible methods of credit assignment in the brain.
arXiv Detail & Related papers (2021-06-30T22:53:03Z) - Self-adjointness in Quantum Mechanics: a pedagogical path [77.34726150561087]
This paper aims to make quantum observables emerge as necessarily self-adjoint, and not merely hermitian operators.
Next to the central core of our line of reasoning, the necessity of a non-trivial declaration of a domain to associate with the formal action of an observable.
arXiv Detail & Related papers (2020-12-28T21:19:33Z) - Unscrambling the omelette of causation and inference: The framework of
causal-inferential theories [0.0]
We introduce the notion of a causal-inferential theory using a process-theoretic formalism.
Recasting the notions of operational and realist theories in this mold clarifies what a realist account of an experiment offers beyond an operational account.
We argue that if one can identify axioms for a realist causal-inferential theory such that the notions of causation and inference can differ from their conventional (classical) interpretations, then one has the means of defining an intrinsically quantum notion of realism.
arXiv Detail & Related papers (2020-09-07T17:58:22Z) - Formalizing Falsification for Theories of Consciousness Across
Computational Hierarchies [0.0]
Integrated Information Theory (IIT) is widely regarded as the preeminent theory of consciousness.
Epistemological issues in the form of the "unfolding argument" have provided a refutation of IIT.
We show how IIT is simultaneously falsified at the finite-state automaton level and unfalsifiable at the state automaton level.
arXiv Detail & Related papers (2020-06-12T18:05:46Z) - A Non-equilibrium Thermodynamic Framework of Consciousness [0.0]
We use functionalist and causal structure theories to motivate a new non-equilibrium thermodynamic framework of consciousness.
The main hypothesis in this paper will be two thermodynamic conditions that a system will have to satisfy in order to be 'conscious'
arXiv Detail & Related papers (2020-05-04T20:01:53Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.