A Message Passing Realization of Expected Free Energy Minimization

• URL: http://arxiv.org/abs/2508.02197v1
• Date: Mon, 04 Aug 2025 08:48:37 GMT
• Title: A Message Passing Realization of Expected Free Energy Minimization
• Authors: Wouter W. L. Nuijten, Mykola Lukashchuk, Thijs van de Laar, Bert de Vries,
• Abstract summary: We present a message passing approach to Expected Free Energy (EFE) on factor graphs.<n>We transform a solvable search problem into a tractable inference problem through standard variational techniques.<n>Applying our message passing method to factorized state-space models enables efficient policy inference.
• Score: 2.099922236065961
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a message passing approach to Expected Free Energy (EFE) minimization on factor graphs, based on the theory introduced in arXiv:2504.14898. By reformulating EFE minimization as Variational Free Energy minimization with epistemic priors, we transform a combinatorial search problem into a tractable inference problem solvable through standard variational techniques. Applying our message passing method to factorized state-space models enables efficient policy inference. We evaluate our method on environments with epistemic uncertainty: a stochastic gridworld and a partially observable Minigrid task. Agents using our approach consistently outperform conventional KL-control agents on these tasks, showing more robust planning and efficient exploration under uncertainty. In the stochastic gridworld environment, EFE-minimizing agents avoid risky paths, while in the partially observable minigrid setting, they conduct more systematic information-seeking. This approach bridges active inference theory with practical implementations, providing empirical evidence for the efficiency of epistemic priors in artificial agents.

Related papers

• Robust Invariant Representation Learning by Distribution Extrapolation [3.5051814539447474]
  Invariant risk minimization (IRM) aims to enable out-of-distribution generalization in deep learning.<n>Existing approaches -- including IRMv1 -- adopt penalty-based single-level approximations.<n>A novel framework is proposed that enhances environmental diversity by augmenting the IRM penalty through synthetic distributional shifts.
  arXiv  Detail & Related papers  (2025-05-22T02:03:34Z)
• Minimum-Excess-Work Guidance [17.15668604906196]
  We propose a regularization framework for guiding pre-trained probability flow generative models.<n>Our approach enables efficient guidance in sparse-data regimes common to scientific applications.<n>We demonstrate the framework's versatility on a coarse-grained protein model.
  arXiv  Detail & Related papers  (2025-05-19T17:19:43Z)
• Expected Free Energy-based Planning as Variational Inference [3.559846046435839]
  We show that EFE-based planning arises naturally from minimizing a variational free energy functional on a generative model.<n>This result reinforces theoretical consistency with the Free Energy Principle by casting planning under uncertainty itself as a form of variational inference.<n>Our formulation yields policies that jointly support goal achievement and information gain, while incorporating a complexity term that accounts for bounded computational resources.
  arXiv  Detail & Related papers  (2025-04-21T07:09:05Z)
• Free Energy Projective Simulation (FEPS): Active inference with interpretability [40.11095094521714]
  Free Energy Projective Simulation (FEP) and active inference (AIF) have achieved many successes. Recent work has focused on improving such agents' performance in complex environments by incorporating the latest machine learning techniques. We introduce Free Energy Projective Simulation (FEPS) to model agents in an interpretable way without deep neural networks.
  arXiv  Detail & Related papers  (2024-11-22T15:01:44Z)
• Energy-Guided Continuous Entropic Barycenter Estimation for General Costs [95.33926437521046]
  We propose a novel algorithm for approximating the continuous Entropic OT (EOT) barycenter for arbitrary OT cost functions.<n>Our approach is built upon the dual reformulation of the EOT problem based on weak OT.
  arXiv  Detail & Related papers  (2023-10-02T11:24:36Z)
• Sampling with Mollified Interaction Energy Descent [57.00583139477843]
  We present a new optimization-based method for sampling called mollified interaction energy descent (MIED) MIED minimizes a new class of energies on probability measures called mollified interaction energies (MIEs) We show experimentally that for unconstrained sampling problems our algorithm performs on par with existing particle-based algorithms like SVGD.
  arXiv  Detail & Related papers  (2022-10-24T16:54:18Z)
• Latent State Marginalization as a Low-cost Approach for Improving Exploration [79.12247903178934]
  We propose the adoption of latent variable policies within the MaxEnt framework. We show that latent variable policies naturally emerges under the use of world models with a latent belief state. We experimentally validate our method on continuous control tasks, showing that effective marginalization can lead to better exploration and more robust training.
  arXiv  Detail & Related papers  (2022-10-03T15:09:12Z)
• Shortest-Path Constrained Reinforcement Learning for Sparse Reward Tasks [59.419152768018506]
  We show that any optimal policy necessarily satisfies the k-SP constraint. We propose a novel cost function that penalizes the policy violating SP constraint, instead of completely excluding it. Our experiments on MiniGrid, DeepMind Lab, Atari, and Fetch show that the proposed method significantly improves proximal policy optimization (PPO)
  arXiv  Detail & Related papers  (2021-07-13T21:39:21Z)
• Strictly Batch Imitation Learning by Energy-based Distribution Matching [104.33286163090179]
  Consider learning a policy purely on the basis of demonstrated behavior -- that is, with no access to reinforcement signals, no knowledge of transition dynamics, and no further interaction with the environment. One solution is simply to retrofit existing algorithms for apprenticeship learning to work in the offline setting. But such an approach leans heavily on off-policy evaluation or offline model estimation, and can be indirect and inefficient. We argue that a good solution should be able to explicitly parameterize a policy, implicitly learn from rollout dynamics, and operate in an entirely offline fashion.
  arXiv  Detail & Related papers  (2020-06-25T03:27:59Z)
• Risk-Aware Energy Scheduling for Edge Computing with Microgrid: A Multi-Agent Deep Reinforcement Learning Approach [82.6692222294594]
  We study a risk-aware energy scheduling problem for a microgrid-powered MEC network. We derive the solution by applying a multi-agent deep reinforcement learning (MADRL)-based advantage actor-critic (A3C) algorithm with shared neural networks.
  arXiv  Detail & Related papers  (2020-02-21T02:14:38Z)
• Targeted free energy estimation via learned mappings [66.20146549150475]
  Free energy perturbation (FEP) was proposed by Zwanzig more than six decades ago as a method to estimate free energy differences. FEP suffers from a severe limitation: the requirement of sufficient overlap between distributions. One strategy to mitigate this problem, called Targeted Free Energy Perturbation, uses a high-dimensional mapping in configuration space to increase overlap.
  arXiv  Detail & Related papers  (2020-02-12T11:10:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.