A Mathematical Walkthrough and Discussion of the Free Energy Principle

• URL: http://arxiv.org/abs/2108.13343v1
• Date: Mon, 30 Aug 2021 16:11:49 GMT
• Title: A Mathematical Walkthrough and Discussion of the Free Energy Principle
• Authors: Beren Millidge, Anil Seth, Christopher L Buckley
• Abstract summary: The Free-Energy-Principle (FEP) is an influential and controversial theory which postulates a connection between the thermodynamics of self-organization and learning through variational inference. FEP has been applied extensively in neuroscience, and is beginning to make inroads in machine learning by spurring the construction of novel and powerful algorithms by which action, perception, and learning can all be unified under a single objective. Here, we aim to provide a mathematically detailed, yet intuitive walk-through of the formulation and central claims of the FEP while also providing a discussion of the assumptions necessary and potential limitations of the theory.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The Free-Energy-Principle (FEP) is an influential and controversial theory which postulates a deep and powerful connection between the stochastic thermodynamics of self-organization and learning through variational inference. Specifically, it claims that any self-organizing system which can be statistically separated from its environment, and which maintains itself at a non-equilibrium steady state, can be construed as minimizing an information-theoretic functional -- the variational free energy -- and thus performing variational Bayesian inference to infer the hidden state of its environment. This principle has also been applied extensively in neuroscience, and is beginning to make inroads in machine learning by spurring the construction of novel and powerful algorithms by which action, perception, and learning can all be unified under a single objective. While its expansive and often grandiose claims have spurred significant debates in both philosophy and theoretical neuroscience, the mathematical depth and lack of accessible introductions and tutorials for the core claims of the theory have often precluded a deep understanding within the literature. Here, we aim to provide a mathematically detailed, yet intuitive walk-through of the formulation and central claims of the FEP while also providing a discussion of the assumptions necessary and potential limitations of the theory. Additionally, since the FEP is a still a living theory, subject to internal controversy, change, and revision, we also present a detailed appendix highlighting and condensing current perspectives as well as controversies about the nature, applicability, and the mathematical assumptions and formalisms underlying the FEP.

Related papers

• On Championing Foundation Models: From Explainability to Interpretability [48.2313835471321]
  This survey aims to review interpretable methods that comply with the aforementioned principles and have been successfully applied to black-box foundation models. The methods are rooted in machine learning theory, covering the analysis of generalization performance, expressive capability, and dynamic behavior. They provide a thorough interpretation of the entire workflow of FMs, ranging from the inference capability and training dynamics to their ethical implications.
  arXiv  Detail & Related papers  (2024-10-15T09:48:03Z)
• Relaxation of first-class constraints and the quantization of gauge theories: from "matter without matter" to the reappearance of time in quantum gravity [72.27323884094953]
  We make a conceptual overview of an approach to the initial-value problem in canonical gauge theories. We stress how the first-class phase-space constraints may be relaxed if we interpret them as fixing the values of new degrees of freedom.
  arXiv  Detail & Related papers  (2024-02-19T19:00:02Z)
• Acquiring and Modelling Abstract Commonsense Knowledge via Conceptualization [49.00409552570441]
  We study the role of conceptualization in commonsense reasoning, and formulate a framework to replicate human conceptual induction. We apply the framework to ATOMIC, a large-scale human-annotated CKG, aided by the taxonomy Probase.
  arXiv  Detail & Related papers  (2022-06-03T12:24:49Z)
• Fact-nets: towards a mathematical framework for relational quantum mechanics [0.0]
  The relational interpretation of quantum mechanics (RQM) has received a growing interest since its first formulation in 1996. This paper proposes a radical reformulation of the mathematical framework of quantum mechanics which is relational from the start: fact-nets.
  arXiv  Detail & Related papers  (2022-04-01T10:27:38Z)
• Active Inference in Robotics and Artificial Agents: Survey and Challenges [51.29077770446286]
  We review the state-of-the-art theory and implementations of active inference for state-estimation, control, planning and learning. We showcase relevant experiments that illustrate its potential in terms of adaptation, generalization and robustness.
  arXiv  Detail & Related papers  (2021-12-03T12:10:26Z)
• Quantum realism: axiomatization and quantification [77.34726150561087]
  We build an axiomatization for quantum realism -- a notion of realism compatible with quantum theory. We explicitly construct some classes of entropic quantifiers that are shown to satisfy almost all of the proposed axioms.
  arXiv  Detail & Related papers  (2021-10-10T18:08:42Z)
• Applications of the Free Energy Principle to Machine Learning and Neuroscience [0.0]
  We explore and apply methods inspired by the free energy principle to two important areas in machine learning and neuroscience. We focus on predictive coding, a neurobiologically plausible process theory derived from the free energy principle. Secondly, we study active inference, a neurobiologically grounded account of action through variational message passing. Finally, we investigate biologically plausible methods of credit assignment in the brain.
  arXiv  Detail & Related papers  (2021-06-30T22:53:03Z)
• Self-adjointness in Quantum Mechanics: a pedagogical path [77.34726150561087]
  This paper aims to make quantum observables emerge as necessarily self-adjoint, and not merely hermitian operators. Next to the central core of our line of reasoning, the necessity of a non-trivial declaration of a domain to associate with the formal action of an observable.
  arXiv  Detail & Related papers  (2020-12-28T21:19:33Z)
• Unscrambling the omelette of causation and inference: The framework of causal-inferential theories [0.0]
  We introduce the notion of a causal-inferential theory using a process-theoretic formalism. Recasting the notions of operational and realist theories in this mold clarifies what a realist account of an experiment offers beyond an operational account. We argue that if one can identify axioms for a realist causal-inferential theory such that the notions of causation and inference can differ from their conventional (classical) interpretations, then one has the means of defining an intrinsically quantum notion of realism.
  arXiv  Detail & Related papers  (2020-09-07T17:58:22Z)
• Formalizing Falsification for Theories of Consciousness Across Computational Hierarchies [0.0]
  Integrated Information Theory (IIT) is widely regarded as the preeminent theory of consciousness. Epistemological issues in the form of the "unfolding argument" have provided a refutation of IIT. We show how IIT is simultaneously falsified at the finite-state automaton level and unfalsifiable at the state automaton level.
  arXiv  Detail & Related papers  (2020-06-12T18:05:46Z)
• A Non-equilibrium Thermodynamic Framework of Consciousness [0.0]
  We use functionalist and causal structure theories to motivate a new non-equilibrium thermodynamic framework of consciousness. The main hypothesis in this paper will be two thermodynamic conditions that a system will have to satisfy in order to be 'conscious'
  arXiv  Detail & Related papers  (2020-05-04T20:01:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.