Active Inference and Epistemic Value in Graphical Models

• URL: http://arxiv.org/abs/2109.00541v1
• Date: Wed, 1 Sep 2021 16:43:35 GMT
• Title: Active Inference and Epistemic Value in Graphical Models
• Authors: Thijs van de Laar, Magnus Koudahl, Bart van Erp, Bert de Vries
• Abstract summary: The Free Energy Principle (FEP) postulates that biological agents perceive and interact with their environment in order to minimize a Variational Free Energy (VFE) with respect to a generative model of their environment. This paper approaches epistemic behavior from a constrained Bethe Free Energy (CBFE) perspective. We illustrate resulting behavior of the CBFE by planning and interacting with a simulated T-maze environment.
• Score: 3.9457043990895904
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The Free Energy Principle (FEP) postulates that biological agents perceive and interact with their environment in order to minimize a Variational Free Energy (VFE) with respect to a generative model of their environment. The inference of a policy (future control sequence) according to the FEP is known as Active Inference (AIF). The AIF literature describes multiple VFE objectives for policy planning that lead to epistemic (information-seeking) behavior. However, most objectives have limited modeling flexibility. This paper approaches epistemic behavior from a constrained Bethe Free Energy (CBFE) perspective. Crucially, variational optimization of the CBFE can be expressed in terms of message passing on free-form generative models. The key intuition behind the CBFE is that we impose a point-mass constraint on predicted outcomes, which explicitly encodes the assumption that the agent will make observations in the future. We interpret the CBFE objective in terms of its constituent behavioral drives. We then illustrate resulting behavior of the CBFE by planning and interacting with a simulated T-maze environment. Simulations for the T-maze task illustrate how the CBFE agent exhibits an epistemic drive, and actively plans ahead to account for the impact of predicted outcomes. Compared to an EFE agent, the CBFE agent incurs expected reward in significantly more environmental scenarios. We conclude that CBFE optimization by message passing suggests a general mechanism for epistemic-aware AIF in free-form generative models.

Related papers

• Free Energy Projective Simulation (FEPS): Active inference with interpretability [40.11095094521714]
  Free Energy Projective Simulation (FEP) and active inference (AIF) have achieved many successes. Recent work has focused on improving such agents' performance in complex environments by incorporating the latest machine learning techniques. We introduce Free Energy Projective Simulation (FEPS) to model agents in an interpretable way without deep neural networks.
  arXiv  Detail & Related papers  (2024-11-22T15:01:44Z)
• R-AIF: Solving Sparse-Reward Robotic Tasks from Pixels with Active Inference and World Models [50.19174067263255]
  We introduce prior preference learning techniques and self-revision schedules to help the agent excel in sparse-reward, continuous action, goal-based robotic control POMDP environments. We show that our agents offer improved performance over state-of-the-art models in terms of cumulative rewards, relative stability, and success rate.
  arXiv  Detail & Related papers  (2024-09-21T18:32:44Z)
• Value of Information and Reward Specification in Active Inference and POMDPs [7.120454740315046]
  Expected free energy (EFE) is a central quantity in active inference. We show that EFE approximates the Bayes optimal RL policy via information value.
  arXiv  Detail & Related papers  (2024-08-13T00:32:05Z)
• Flexible and Robust Counterfactual Explanations with Minimal Satisfiable Perturbations [56.941276017696076]
  We propose a conceptually simple yet effective solution named Counterfactual Explanations with Minimal Satisfiable Perturbations (CEMSP) CEMSP constrains changing values of abnormal features with the help of their semantically meaningful normal ranges. Compared to existing methods, we conduct comprehensive experiments on both synthetic and real-world datasets to demonstrate that our method provides more robust explanations while preserving flexibility.
  arXiv  Detail & Related papers  (2023-09-09T04:05:56Z)
• For Better or Worse: The Impact of Counterfactual Explanations' Directionality on User Behavior in xAI [6.883906273999368]
  Counterfactual explanations (CFEs) are a popular approach in explainable artificial intelligence (xAI) CFEs describe a scenario that is better than the factual state (upward CFE), or a scenario that is worse than the factual state (downward CFE) This study compares the impact of CFE directionality on behavior and experience of participants tasked to extract new knowledge from an automated system.
  arXiv  Detail & Related papers  (2023-06-13T09:16:38Z)
• Realising Synthetic Active Inference Agents, Part II: Variational Message Updates [2.2940141855172036]
  Active Inference (AIF) is a corollary of the Free Energy Principle (FEP) We describe a scalable, epistemic approach to synthetic AIF, by message passing on free-form Forney-style Factor Graphs (FFGs) With a full message passing account of synthetic AIF agents, it becomes possible to derive and reuse message updates across models.
  arXiv  Detail & Related papers  (2023-06-05T09:29:46Z)
• A Neural Active Inference Model of Perceptual-Motor Learning [62.39667564455059]
  The active inference framework (AIF) is a promising new computational framework grounded in contemporary neuroscience. In this study, we test the ability for the AIF to capture the role of anticipation in the visual guidance of action in humans. We present a novel formulation of the prior function that maps a multi-dimensional world-state to a uni-dimensional distribution of free-energy.
  arXiv  Detail & Related papers  (2022-11-16T20:00:38Z)
• Active inference, Bayesian optimal design, and expected utility [1.433758865948252]
  We describe how active inference combines Bayesian decision theory and optimal Bayesian design principles to minimize expected free energy. It is this aspect of active inference that allows for the natural emergence of information-seeking behavior. Our Tmaze simulations show optimizing expected free energy produces goal-directed information-seeking behavior while optimizing expected utility induces purely exploitive behavior.
  arXiv  Detail & Related papers  (2021-09-21T20:56:32Z)
• Instance-Aware Predictive Navigation in Multi-Agent Environments [93.15055834395304]
  We propose an Instance-Aware Predictive Control (IPC) approach, which forecasts interactions between agents as well as future scene structures. We adopt a novel multi-instance event prediction module to estimate the possible interaction among agents in the ego-centric view. We design a sequential action sampling strategy to better leverage predicted states on both scene-level and instance-level.
  arXiv  Detail & Related papers  (2021-01-14T22:21:25Z)
• Exploiting Submodular Value Functions For Scaling Up Active Perception [60.81276437097671]
  In active perception tasks, agent aims to select sensory actions that reduce uncertainty about one or more hidden variables. Partially observable Markov decision processes (POMDPs) provide a natural model for such problems. As the number of sensors available to the agent grows, the computational cost of POMDP planning grows exponentially.
  arXiv  Detail & Related papers  (2020-09-21T09:11:36Z)
• Whence the Expected Free Energy? [68.8204255655161]
  We show that the Expected Free Energy (EFE) is not simply "the free energy in the future" We then develop a novel objective, the Free-Energy of the Expected Future (FEEF)
  arXiv  Detail & Related papers  (2020-04-17T09:06:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.