Whence the Expected Free Energy?

• URL: http://arxiv.org/abs/2004.08128v5
• Date: Mon, 28 Sep 2020 21:04:16 GMT
• Title: Whence the Expected Free Energy?
• Authors: Beren Millidge, Alexander Tschantz, Christopher L Buckley
• Abstract summary: We show that the Expected Free Energy (EFE) is not simply "the free energy in the future" We then develop a novel objective, the Free-Energy of the Expected Future (FEEF)
• Score: 68.8204255655161
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The Expected Free Energy (EFE) is a central quantity in the theory of active inference. It is the quantity that all active inference agents are mandated to minimize through action, and its decomposition into extrinsic and intrinsic value terms is key to the balance of exploration and exploitation that active inference agents evince. Despite its importance, the mathematical origins of this quantity and its relation to the Variational Free Energy (VFE) remain unclear. In this paper, we investigate the origins of the EFE in detail and show that it is not simply "the free energy in the future". We present a functional that we argue is the natural extension of the VFE, but which actively discourages exploratory behaviour, thus demonstrating that exploration does not directly follow from free energy minimization into the future. We then develop a novel objective, the Free-Energy of the Expected Future (FEEF), which possesses both the epistemic component of the EFE as well as an intuitive mathematical grounding as the divergence between predicted and desired futures.

Related papers

• Value of Information and Reward Specification in Active Inference and POMDPs [7.120454740315046]
  Expected free energy (EFE) is a central quantity in active inference. We show that EFE approximates the Bayes optimal RL policy via information value.
  arXiv  Detail & Related papers  (2024-08-13T00:32:05Z)
• Reframing the Expected Free Energy: Four Formulations and a Unification [3.9121134770873733]
  Active inference is based on the expected free energy. This paper seek to formalize the problem of deriving these formulations from a single root expected free energy definition.
  arXiv  Detail & Related papers  (2024-02-22T11:38:43Z)
• A Neural Active Inference Model of Perceptual-Motor Learning [62.39667564455059]
  The active inference framework (AIF) is a promising new computational framework grounded in contemporary neuroscience. In this study, we test the ability for the AIF to capture the role of anticipation in the visual guidance of action in humans. We present a novel formulation of the prior function that maps a multi-dimensional world-state to a uni-dimensional distribution of free-energy.
  arXiv  Detail & Related papers  (2022-11-16T20:00:38Z)
• Variational Inference with Holder Bounds [68.8008396694788]
  We present a careful analysis of the thermodynamic variational objective (TVO) We reveal how the pathological geometry of thermodynamic curves negatively affects TVO. This motivates our new VI objectives, named the Holder bounds, which flatten the thermodynamic curves and promise to achieve a one-step approximation of the exact marginal log-likelihood.
  arXiv  Detail & Related papers  (2021-11-04T15:35:47Z)
• Active inference, Bayesian optimal design, and expected utility [1.433758865948252]
  We describe how active inference combines Bayesian decision theory and optimal Bayesian design principles to minimize expected free energy. It is this aspect of active inference that allows for the natural emergence of information-seeking behavior. Our Tmaze simulations show optimizing expected free energy produces goal-directed information-seeking behavior while optimizing expected utility induces purely exploitive behavior.
  arXiv  Detail & Related papers  (2021-09-21T20:56:32Z)
• Active Inference and Epistemic Value in Graphical Models [3.9457043990895904]
  The Free Energy Principle (FEP) postulates that biological agents perceive and interact with their environment in order to minimize a Variational Free Energy (VFE) with respect to a generative model of their environment. This paper approaches epistemic behavior from a constrained Bethe Free Energy (CBFE) perspective. We illustrate resulting behavior of the CBFE by planning and interacting with a simulated T-maze environment.
  arXiv  Detail & Related papers  (2021-09-01T16:43:35Z)
• The problem of engines in statistical physics [62.997667081978825]
  Engines are open systems that can generate work cyclically, at the expense of an external disequilibrium. Recent advances in the theory of open quantum systems point to a more realistic description of autonomous engines. We show how the external loading force and the thermal noise may be incorporated into the relevant equations of motion.
  arXiv  Detail & Related papers  (2021-08-17T03:59:09Z)
• Collective effects on the performance and stability of quantum heat engines [62.997667081978825]
  Recent predictions for quantum-mechanical enhancements in the operation of small heat engines have raised renewed interest. One essential question is whether collective effects may help to carry enhancements over larger scales. We study how power, efficiency and constancy scale with the number of spins composing the engine.
  arXiv  Detail & Related papers  (2021-06-25T18:00:07Z)
• Sophisticated Inference [8.145323363883234]
  Active inference offers a first principle account of sentient behaviour. It replaces value functions with functionals of (Bayesian) beliefs. In this paper, we consider a sophisticated kind of active inference.
  arXiv  Detail & Related papers  (2020-06-07T11:18:58Z)
• Maximizing Information Gain in Partially Observable Environments via Prediction Reward [64.24528565312463]
  This paper tackles the challenge of using belief-based rewards for a deep RL agent. We derive the exact error between negative entropy and the expected prediction reward. This insight provides theoretical motivation for several fields using prediction rewards.
  arXiv  Detail & Related papers  (2020-05-11T08:13:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.