Active inference, Bayesian optimal design, and expected utility

• URL: http://arxiv.org/abs/2110.04074v1
• Date: Tue, 21 Sep 2021 20:56:32 GMT
• Title: Active inference, Bayesian optimal design, and expected utility
• Authors: Noor Sajid, Lancelot Da Costa, Thomas Parr, Karl Friston
• Abstract summary: We describe how active inference combines Bayesian decision theory and optimal Bayesian design principles to minimize expected free energy. It is this aspect of active inference that allows for the natural emergence of information-seeking behavior. Our Tmaze simulations show optimizing expected free energy produces goal-directed information-seeking behavior while optimizing expected utility induces purely exploitive behavior.
• Score: 1.433758865948252
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Active inference, a corollary of the free energy principle, is a formal way of describing the behavior of certain kinds of random dynamical systems that have the appearance of sentience. In this chapter, we describe how active inference combines Bayesian decision theory and optimal Bayesian design principles under a single imperative to minimize expected free energy. It is this aspect of active inference that allows for the natural emergence of information-seeking behavior. When removing prior outcomes preferences from expected free energy, active inference reduces to optimal Bayesian design, i.e., information gain maximization. Conversely, active inference reduces to Bayesian decision theory in the absence of ambiguity and relative risk, i.e., expected utility maximization. Using these limiting cases, we illustrate how behaviors differ when agents select actions that optimize expected utility, expected information gain, and expected free energy. Our T-maze simulations show optimizing expected free energy produces goal-directed information-seeking behavior while optimizing expected utility induces purely exploitive behavior and maximizing information gain engenders intrinsically motivated behavior.

Related papers

• Enhancing Population-based Search with Active Inference [0.0]
  This paper proposes the integration of Active Inference into population-based metaheuristics to enhance performance. Experimental results indicate that Active Inference can yield some improved solutions with only a marginal increase in computational cost.
  arXiv  Detail & Related papers  (2024-08-18T17:21:21Z)
• Value of Information and Reward Specification in Active Inference and POMDPs [7.120454740315046]
  Expected free energy (EFE) is a central quantity in active inference. We show that EFE approximates the Bayes optimal RL policy via information value.
  arXiv  Detail & Related papers  (2024-08-13T00:32:05Z)
• Confronting Reward Overoptimization for Diffusion Models: A Perspective of Inductive and Primacy Biases [76.9127853906115]
  Bridging the gap between diffusion models and human preferences is crucial for their integration into practical generative. We propose Temporal Diffusion Policy Optimization with critic active neuron Reset (TDPO-R), a policy gradient algorithm that exploits the temporal inductive bias of diffusion models. Empirical results demonstrate the superior efficacy of our methods in mitigating reward overoptimization.
  arXiv  Detail & Related papers  (2024-02-13T15:55:41Z)
• Modeling arousal potential of epistemic emotions using Bayesian information gain: Inquiry cycle driven by free energy fluctuations [0.0]
  Epistem emotions, such as curiosity and interest, drive the inquiry process. Two types of information gain generated by the principle of free energy and divergence: Kullback-Leibler(KLD) We analyzed the effects of prediction uncertainty (prior variance) and observation uncertainty (likelihood variance) on the peaks of the information gain function as optimal surprises.
  arXiv  Detail & Related papers  (2023-12-14T02:59:20Z)
• Model-based Causal Bayesian Optimization [74.78486244786083]
  We introduce the first algorithm for Causal Bayesian Optimization with Multiplicative Weights (CBO-MW) We derive regret bounds for CBO-MW that naturally depend on graph-related quantities. Our experiments include a realistic demonstration of how CBO-MW can be used to learn users' demand patterns in a shared mobility system.
  arXiv  Detail & Related papers  (2023-07-31T13:02:36Z)
• Prediction-Oriented Bayesian Active Learning [51.426960808684655]
  Expected predictive information gain (EPIG) is an acquisition function that measures information gain in the space of predictions rather than parameters. EPIG leads to stronger predictive performance compared with BALD across a range of datasets and models.
  arXiv  Detail & Related papers  (2023-04-17T10:59:57Z)
• A Neural Active Inference Model of Perceptual-Motor Learning [62.39667564455059]
  The active inference framework (AIF) is a promising new computational framework grounded in contemporary neuroscience. In this study, we test the ability for the AIF to capture the role of anticipation in the visual guidance of action in humans. We present a novel formulation of the prior function that maps a multi-dimensional world-state to a uni-dimensional distribution of free-energy.
  arXiv  Detail & Related papers  (2022-11-16T20:00:38Z)
• Generalizing Bayesian Optimization with Decision-theoretic Entropies [102.82152945324381]
  We consider a generalization of Shannon entropy from work in statistical decision theory. We first show that special cases of this entropy lead to popular acquisition functions used in BO procedures. We then show how alternative choices for the loss yield a flexible family of acquisition functions.
  arXiv  Detail & Related papers  (2022-10-04T04:43:58Z)
• Preference Enhanced Social Influence Modeling for Network-Aware Cascade Prediction [59.221668173521884]
  We propose a novel framework to promote cascade size prediction by enhancing the user preference modeling. Our end-to-end method makes the user activating process of information diffusion more adaptive and accurate.
  arXiv  Detail & Related papers  (2022-04-18T09:25:06Z)
• Goal-Directed Planning by Reinforcement Learning and Active Inference [16.694117274961016]
  We propose a novel computational framework of decision making with Bayesian inference. Goal-directed behavior is determined from the posterior distribution of $z$ by planning. We demonstrate the effectiveness of the proposed framework by experiments in a sensorimotor navigation task with camera observations and continuous motor actions.
  arXiv  Detail & Related papers  (2021-06-18T06:41:01Z)
• Reward Maximisation through Discrete Active Inference [1.2074552857379273]
  We show how and when active inference agents perform actions that are optimal for maximising reward. We show the conditions under which active inference produces the optimal solution to the Bellman equation. We append the analysis with a discussion of the broader relationship between active inference and reinforcement learning.
  arXiv  Detail & Related papers  (2020-09-17T07:13:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.