Expected Free Energy-based Planning as Variational Inference

• URL: http://arxiv.org/abs/2606.20658v1
• Date: Tue, 09 Jun 2026 08:09:48 GMT
• Title: Expected Free Energy-based Planning as Variational Inference
• Abstract summary: We show that EFE-based planning can be formulated as Variational Free Energy minimization.<n>Our main result demonstrates that minimizing a Variational Free Energy functional with appropriately chosen priors yields a decomposition into expected plan costs.
• Score: 1.2731778729620478
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Planning under uncertainty requires agents to balance goal achievement with information gathering. Active inference addresses this through the Expected Free Energy (EFE), a cost function that unifies instrumental and epistemic objectives. However, existing EFE-based methods typically employ specialized optimization procedures that are difficult to extend or analyze. In this paper, we show that EFE-based planning can be formulated as Variational Free Energy minimization on a generative model augmented with epistemic priors. Our main result demonstrates that minimizing a Variational Free Energy functional with appropriately chosen priors yields a decomposition into expected plan costs (the EFE) plus a complexity term. This formulation reinforces theoretical consistency with the Free Energy Principle by casting planning as the same inferential process that governs perception and learning. We validate our approach on three environments of increasing complexity: a deterministic T-maze, a stochastic Reactivity Maze, and a partially observable MiniGrid DoorKey-8x8 environment. The experiments demonstrate that the epistemic priors induce information-seeking behavior, that the variational formulation yields policy-based inference outperforming plan-based methods under stochastic transitions, and that temporal factorization enables scalability to environments where existing tabular active inference methods cannot operate.

Related papers

• Action-BED: Task-Driven Bayesian Experimental Design with Singly Intractable Objectives [12.963546988204875]
  We show that an expected future loss (EFL) on downstream actions can provide a simple and naturally task-driven framework for design policies.<n>We then show that all such EFLs can be rearranged into singly intractable objectives that can be jointly optimised with respect to both the design policy and a downstream action policy.
  arXiv  Detail & Related papers  (2026-06-22T17:48:07Z)
• What Type of Inference is Active Inference? [1.1654568410166082]
  Active inference casts decision-making as inference, with the Expected Free Energy (EFE) unifying goal-directed and information-seeking behavior.<n>Recent work showed that EFE can be written as Variational Free Energy (VFE) minimization on a generative model augmented with epistemic priors.<n>We show that proper EFE-based planning requires combining these epistemic corrections with a planning correction that turns marginal inference into policy optimization.<n>This clarifies which corrections are needed for cross-entropy planning and for full EFE-based planning.
  arXiv  Detail & Related papers  (2026-06-03T14:24:53Z)
• Information as Maximum-Caliber Deviation: A bridge between Integrated Information Theory and the Free Energy Principle [51.56484100374058]
  This work proposes that information can be defined as the deviation $$ of realized dynamics from a constrained maximum-caliber (MaxCal) path ensemble over a finite time horizon.<n>Under this definition, each of the cause/effect repertoires central to IIT 3.0 emerge directly from MaxCal variational principles.<n>Information $$ is shown to be equivalent to prediction error under accompanying predictive coding models.
  arXiv  Detail & Related papers  (2026-05-03T07:22:55Z)
• On the Entropy Dynamics in Reinforcement Fine-Tuning of Large Language Models [54.61810451777578]
  Entropy serves as a critical metric for measuring the diversity of outputs generated by large language models.<n>Recent studies increasingly focus on monitoring and adjusting entropy to better balance exploration and exploitation in reinforcement fine-tuning.
  arXiv  Detail & Related papers  (2026-02-03T11:14:58Z)
• Active Inference is a Subtype of Variational Inference [0.0]
  We build on theory recasting EFE minimization as variational inference, formally unifying it with Planning-as-Inference and showing the drive as a unique entropic contribution.<n>Our main contribution is a novel message-passing scheme for this unified objective, enabling scalable Active Inference in factored-state MDPs and overcoming high-dimensional planning intractability.
  arXiv  Detail & Related papers  (2025-11-24T10:14:09Z)
• A Message Passing Realization of Expected Free Energy Minimization [2.099922236065961]
  We present a message passing approach to Expected Free Energy (EFE) on factor graphs.<n>We transform a solvable search problem into a tractable inference problem through standard variational techniques.<n>Applying our message passing method to factorized state-space models enables efficient policy inference.
  arXiv  Detail & Related papers  (2025-08-04T08:48:37Z)
• Expected Free Energy-based Planning as Variational Inference [2.5935401957183846]
  We show that EFE-based planning arises naturally from minimizing a variational free energy functional on a generative model.<n>This result reinforces theoretical consistency with the Free Energy Principle by casting planning under uncertainty itself as a form of variational inference.<n>Our formulation yields policies that jointly support goal achievement and information gain, while incorporating a complexity term that accounts for bounded computational resources.
  arXiv  Detail & Related papers  (2025-04-21T07:09:05Z)
• Model Predictive Task Sampling for Efficient and Robust Adaptation [57.414812940406996]
  We introduce Model Predictive Task Sampling (MPTS), a framework that bridges the task space and adaptation risk distributions.<n>MPTS employs a generative model to characterize the episodic optimization process and predicts task-specific adaptation risk via posterior inference.<n>MPTS seamlessly integrates into zero-shot, few-shot, and supervised finetuning settings.
  arXiv  Detail & Related papers  (2025-01-19T13:14:53Z)
• Entropy-Regularized Token-Level Policy Optimization for Language Agent Reinforcement [67.1393112206885]
  Large Language Models (LLMs) have shown promise as intelligent agents in interactive decision-making tasks. We introduce Entropy-Regularized Token-level Policy Optimization (ETPO), an entropy-augmented RL method tailored for optimizing LLMs at the token level. We assess the effectiveness of ETPO within a simulated environment that models data science code generation as a series of multi-step interactive tasks.
  arXiv  Detail & Related papers  (2024-02-09T07:45:26Z)
• Scalable PAC-Bayesian Meta-Learning via the PAC-Optimal Hyper-Posterior: From Theory to Practice [54.03076395748459]
  A central question in the meta-learning literature is how to regularize to ensure generalization to unseen tasks. We present a generalization bound for meta-learning, which was first derived by Rothfuss et al. We provide a theoretical analysis and empirical case study under which conditions and to what extent these guarantees for meta-learning improve upon PAC-Bayesian per-task learning bounds.
  arXiv  Detail & Related papers  (2022-11-14T08:51:04Z)
• Targeted free energy estimation via learned mappings [66.20146549150475]
  Free energy perturbation (FEP) was proposed by Zwanzig more than six decades ago as a method to estimate free energy differences. FEP suffers from a severe limitation: the requirement of sufficient overlap between distributions. One strategy to mitigate this problem, called Targeted Free Energy Perturbation, uses a high-dimensional mapping in configuration space to increase overlap.
  arXiv  Detail & Related papers  (2020-02-12T11:10:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.