Information as Maximum-Caliber Deviation: A bridge between Integrated Information Theory and the Free Energy Principle

• URL: http://arxiv.org/abs/2605.12536v1
• Date: Sun, 03 May 2026 07:22:55 GMT
• Title: Information as Maximum-Caliber Deviation: A bridge between Integrated Information Theory and the Free Energy Principle
• Abstract summary: This work proposes that information can be defined as the deviation $$ of realized dynamics from a constrained maximum-caliber (MaxCal) path ensemble over a finite time horizon.<n>Under this definition, each of the cause/effect repertoires central to IIT 3.0 emerge directly from MaxCal variational principles.<n>Information $$ is shown to be equivalent to prediction error under accompanying predictive coding models.
• Score: 51.56484100374058
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The Free Energy Principle (FEP) is a leading framework for mathematically modeling self-organization and learning, while Integrated Information Theory (IIT) is a computational ontology of consciousness oriented around irreducible cause and effect. While conceptual unifications have been proposed and appear to be supported by empirical findings, the absence of a rigorous mathematical mapping places upper bounds on their precision and testability. This work proposes that information can be defined as the deviation $ψ$ of realized dynamics from a constrained maximum-caliber (MaxCal) path ensemble over a finite time horizon. Under this definition, each of the cause/effect repertoires central to IIT 3.0 emerge directly from MaxCal variational principles, allowing IIT's phenomenological calculus to be re-derived from constrained entropy-maximization (CMEP). This framework supplies a theoretical bridge to active inference, which is mathematically dual to CMEP under Langevin dynamics, and offers a principled route for extending IIT to new dynamical regimes. When the approach is applied under the Central Limit Theorem (CLT) for Markov chains and via large deviations theory (LDT) to Ising models, information $ψ$ is shown to be equivalent to prediction error under accompanying predictive coding models. This may hold relevance to the ``hill-shaped trajectory'' of $Φ$ observed in neuronal cultures adapting to sensory inputs. Together, these results provide a physically and mathematically grounded rationale for studying the convergence of FEP, IIT, and thermodynamic frameworks of cognition such as recent work grounding consciousness in violations of the Fluctuation-Dissipation Theorem (FDT).

Related papers

• CDFM: Towards a General-Purpose Causal Discovery Foundation Model [62.32138962508216]
  Causal discovery is the process of recovering underlying causal structures from observational data.<n>We formulate the Causal Discovery Foundation Model (CDFM) as a unified, general-purpose framework for zero-shot structural inference.<n>CDFM consistently outperforms traditional algorithms, driving a paradigm shift toward a general-purpose causal discovery foundation model.
  arXiv  Detail & Related papers  (2026-07-13T12:58:28Z)
• OMNIFLOW: A Physics-Grounded Multimodal Agent for Generalized Scientific Reasoning [51.33849811496781]
  Large Language Models (LLMs) have demonstrated exceptional logical reasoning capabilities but frequently struggle with the continuous dynamics by Partial Differential Equations (PDEs)<n>We propose OMNIFLOW, a multimodal-symbolic architecture designed to ground frozen LLMs in fundamental physical laws without requiring domain-specific parameter updates.<n>We evaluate it on a benchmark spanning microscopic turbulence, theoretical Navier-Stokes, and macroscopic global weather forecasting.
  arXiv  Detail & Related papers  (2026-03-16T18:29:01Z)
• Conjugate Learning Theory: Uncovering the Mechanisms of Trainability and Generalization in Deep Neural Networks [0.0]
  We develop a conjugate learning theoretical framework based on convex conjugate duality to characterize this learnability property.<n>We demonstrate that training deep neural networks (DNNs) with mini-batch descent (SGD) achieves global optima of empirical risk.<n>We derive deterministic and probabilistic bounds on generalization error based on conditional generalized entropy measures.
  arXiv  Detail & Related papers  (2026-02-18T04:26:55Z)
• Beyond the Black Box: Theory and Mechanism of Large Language Models [39.10631426330405]
  The rapid emergence of Large Language Models (LLMs) has precipitated a profound paradigm shift in Artificial Intelligence.<n>This survey proposes a unified lifecycle-based taxonomy that organizes the research landscape into six distinct stages: Data Preparation, Model Preparation, Training, Alignment, Inference, and Evaluation.
  arXiv  Detail & Related papers  (2026-01-06T10:45:53Z)
• BEDS : Bayesian Emergent Dissipative Structures : A Formal Framework for Continuous Inference Under Energy Constraints [0.6345523830122167]
  We introduce BEDS, a formal framework for analyzing inference systems that must maintain beliefs continuously under energy constraints.<n>We prove a central result linking energy, precision, and dissipation.<n>We propose the Gdel-Landauer-Prigogine conjecture, suggesting that closure pathologies across formal systems, computation, and thermodynamics share a common structure.
  arXiv  Detail & Related papers  (2026-01-05T18:21:02Z)
• A Generalized Information Bottleneck Theory of Deep Learning [10.454976783057086]
  The Information Bottleneck (IB) principle offers a compelling theoretical framework to understand how neural networks (NNs) learn.<n>We present a framework that reformulates the original IB principle through the lens of synergy.
  arXiv  Detail & Related papers  (2025-09-30T14:38:56Z)
• Learning Truthful Mechanisms without Discretization [6.47853785922226]
  This paper introduces TEDI, a discretization-free algorithm to learn truthful and utility-maximizing mechanisms.<n>We formalize the concept of pricing rules, defined as functions that map outcomes to prices.<n>Based on this concept, we propose a novel menu mechanism, which can be equivalent to a truthful direct mechanism under specific conditions.
  arXiv  Detail & Related papers  (2025-06-28T14:50:29Z)
• The Curse of CoT: On the Limitations of Chain-of-Thought in In-Context Learning [56.574829311863446]
  Chain-of-Thought (CoT) prompting has been widely recognized for its ability to enhance reasoning capabilities in large language models (LLMs)<n>We demonstrate that CoT and its reasoning variants consistently underperform direct answering across varying model scales and benchmark complexities.<n>Our analysis uncovers a fundamental hybrid mechanism of explicit-implicit reasoning driving CoT's performance in pattern-based ICL.
  arXiv  Detail & Related papers  (2025-04-07T13:51:06Z)
• Chaos into Order: Neural Framework for Expected Value Estimation of Stochastic Partial Differential Equations [0.9944647907864256]
  We propose a physics-informed neural framework designed to approximate the expected value of linear partial differential equations (SPDEs)<n>By leveraging randomized sampling of both space-time coordinates and noise realizations during training, LEC trains standard feedforward neural networks to minimize residual loss across multiple samples.<n>We show that the model consistently learns accurate approximations of the expected value of the solution in lower dimensions and a predictable decrease in robustness with increased spatial dimensions.
  arXiv  Detail & Related papers  (2025-02-05T23:27:28Z)
• A PAC-Bayesian Perspective on the Interpolating Information Criterion [54.548058449535155]
  We show how a PAC-Bayes bound is obtained for a general class of models, characterizing factors which influence performance in the interpolating regime. We quantify how the test error for overparameterized models achieving effectively zero training error depends on the quality of the implicit regularization imposed by e.g. the combination of model, parameter-initialization scheme.
  arXiv  Detail & Related papers  (2023-11-13T01:48:08Z)
• Pseudo-Spherical Contrastive Divergence [119.28384561517292]
  We propose pseudo-spherical contrastive divergence (PS-CD) to generalize maximum learning likelihood of energy-based models. PS-CD avoids the intractable partition function and provides a generalized family of learning objectives.
  arXiv  Detail & Related papers  (2021-11-01T09:17:15Z)
• A Mathematical Walkthrough and Discussion of the Free Energy Principle [0.0]
  The Free-Energy-Principle (FEP) is an influential and controversial theory which postulates a connection between the thermodynamics of self-organization and learning through variational inference. FEP has been applied extensively in neuroscience, and is beginning to make inroads in machine learning by spurring the construction of novel and powerful algorithms by which action, perception, and learning can all be unified under a single objective. Here, we aim to provide a mathematically detailed, yet intuitive walk-through of the formulation and central claims of the FEP while also providing a discussion of the assumptions necessary and potential limitations of the theory.
  arXiv  Detail & Related papers  (2021-08-30T16:11:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.