Realising Synthetic Active Inference Agents, Part I: Epistemic Objectives and Graphical Specification Language

• URL: http://arxiv.org/abs/2306.08014v2
• Date: Mon, 16 Oct 2023 09:39:16 GMT
• Title: Realising Synthetic Active Inference Agents, Part I: Epistemic Objectives and Graphical Specification Language
• Authors: Magnus Koudahl, Thijs van de Laar, Bert de Vries
• Abstract summary: This paper is the first in a series of two where we derive a synthetic version of Active Inference on free form factor graphs. We develop Constrained Forney-style Factor Graph notation which permits a fully graphical description of variational inference objectives. We derive an algorithm that permits direct policy inference for AIF agents, circumventing a long standing scaling issue.
• Score: 2.5782420501870296
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The Free Energy Principle (FEP) is a theoretical framework for describing how (intelligent) systems self-organise into coherent, stable structures by minimising a free energy functional. Active Inference (AIF) is a corollary of the FEP that specifically details how systems that are able to plan for the future (agents) function by minimising particular free energy functionals that incorporate information seeking components. This paper is the first in a series of two where we derive a synthetic version of AIF on free form factor graphs. The present paper focuses on deriving a local version of the free energy functionals used for AIF. This enables us to construct a version of AIF which applies to arbitrary graphical models and interfaces with prior work on message passing algorithms. The resulting messages are derived in our companion paper. We also identify a gap in the graphical notation used for factor graphs. While factor graphs are great at expressing a generative model, they have so far been unable to specify the full optimisation problem including constraints. To solve this problem we develop Constrained Forney-style Factor Graph (CFFG) notation which permits a fully graphical description of variational inference objectives. We then proceed to show how CFFG's can be used to reconstruct prior algorithms for AIF as well as derive new ones. The latter is demonstrated by deriving an algorithm that permits direct policy inference for AIF agents, circumventing a long standing scaling issue that has so far hindered the application of AIF in industrial settings. We demonstrate our algorithm on the classic T-maze task and show that it reproduces the information seeking behaviour that is a hallmark feature of AIF.

Related papers

• Autoencoders in Function Space [5.558412940088621]
  This paper introduces function-space versions of the autoencoder (FAE) and variational autoencoder (FVAE) The FAE objective is valid much more broadly, and can be straightforwardly applied to data governed by differential equations. Pairing these objectives with neural operator architectures, which can be evaluated on any mesh, enables new applications of autoencoders to inpainting, superresolution, and generative modelling of scientific data.
  arXiv  Detail & Related papers  (2024-08-02T16:13:51Z)
• Towards Automated Functional Equation Proving: A Benchmark Dataset and A Domain-Specific In-Context Agent [1.006303657343407]
  Automated Theorem Proving (ATP) faces challenges due to its complexity and computational demands. Recent work has explored using Large Language Models (LLMs) for ATP action selection, but these methods can be resource-intensive. This study introduces FEAS, an agent that enhances the COPRA in-context learning framework within Lean.
  arXiv  Detail & Related papers  (2024-07-05T15:59:16Z)
• Federated Knowledge Graph Completion via Latent Embedding Sharing and Tensor Factorization [51.286715478399515]
  Federated Latent Embedding factorization (FLEST) is a novel approach using federated factorization for KG completion. FLEST decomposes the embedding matrix and enables sharing of latent dictionary embeddings to lower privacy risks. Empirical results demonstrate FLEST's effectiveness and efficiency, offering a balanced solution between performance and privacy.
  arXiv  Detail & Related papers  (2023-11-17T06:03:56Z)
• GAFlow: Incorporating Gaussian Attention into Optical Flow [62.646389181507764]
  We push Gaussian Attention (GA) into the optical flow models to accentuate local properties during representation learning. We introduce a novel Gaussian-Constrained Layer (GCL) which can be easily plugged into existing Transformer blocks. For reliable motion analysis, we provide a new Gaussian-Guided Attention Module (GGAM)
  arXiv  Detail & Related papers  (2023-09-28T07:46:01Z)
• FP-IRL: Fokker-Planck-based Inverse Reinforcement Learning -- A Physics-Constrained Approach to Markov Decision Processes [0.5735035463793008]
  Inverse Reinforcement Learning (IRL) is a technique for revealing the rationale underlying the behavior of autonomous agents. IRL seeks to estimate the unknown reward function of a Markov decision process (MDP) from observed agent trajectories. We create a novel IRL algorithm, FP-IRL, which can simultaneously infer the transition and reward functions using only observed trajectories.
  arXiv  Detail & Related papers  (2023-06-17T18:28:03Z)
• Realising Synthetic Active Inference Agents, Part II: Variational Message Updates [2.2940141855172036]
  Active Inference (AIF) is a corollary of the Free Energy Principle (FEP) We describe a scalable, epistemic approach to synthetic AIF, by message passing on free-form Forney-style Factor Graphs (FFGs) With a full message passing account of synthetic AIF agents, it becomes possible to derive and reuse message updates across models.
  arXiv  Detail & Related papers  (2023-06-05T09:29:46Z)
• GIF: A General Graph Unlearning Strategy via Influence Function [63.52038638220563]
  Graph Influence Function (GIF) is a model-agnostic unlearning method that can efficiently and accurately estimate parameter changes in response to a $epsilon$-mass perturbation in deleted data. We conduct extensive experiments on four representative GNN models and three benchmark datasets to justify GIF's superiority in terms of unlearning efficacy, model utility, and unlearning efficiency.
  arXiv  Detail & Related papers  (2023-04-06T03:02:54Z)
• Graph Signal Sampling for Inductive One-Bit Matrix Completion: a Closed-form Solution [112.3443939502313]
  We propose a unified graph signal sampling framework which enjoys the benefits of graph signal analysis and processing. The key idea is to transform each user's ratings on the items to a function (signal) on the vertices of an item-item graph. For the online setting, we develop a Bayesian extension, i.e., BGS-IMC which considers continuous random Gaussian noise in the graph Fourier domain.
  arXiv  Detail & Related papers  (2023-02-08T08:17:43Z)
• Variational Flow Graphical Model [22.610974083362606]
  Variational Graphical Flow (VFG) Model learns the representation of high dimensional data via a message-passing scheme. VFGs produce a representation of the data using a lower dimension, thus overcoming the drawbacks of many flow-based models. In experiments, VFGs achieves improved evidence lower bound (ELBO) and likelihood values on multiple datasets.
  arXiv  Detail & Related papers  (2022-07-06T14:51:03Z)
• Let Invariant Rationale Discovery Inspire Graph Contrastive Learning [98.10268114789775]
  We argue that a high-performing augmentation should preserve the salient semantics of anchor graphs regarding instance-discrimination. We propose a new framework, Rationale-aware Graph Contrastive Learning (RGCL) RGCL uses a rationale generator to reveal salient features about graph instance-discrimination as the rationale, and then creates rationale-aware views for contrastive learning.
  arXiv  Detail & Related papers  (2022-06-16T01:28:40Z)
• Estimating Structural Target Functions using Machine Learning and Influence Functions [103.47897241856603]
  We propose a new framework for statistical machine learning of target functions arising as identifiable functionals from statistical models. This framework is problem- and model-agnostic and can be used to estimate a broad variety of target parameters of interest in applied statistics. We put particular focus on so-called coarsening at random/doubly robust problems with partially unobserved information.
  arXiv  Detail & Related papers  (2020-08-14T16:48:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.