Branching Time Active Inference: the theory and its generality

• URL: http://arxiv.org/abs/2111.11107v1
• Date: Mon, 22 Nov 2021 10:56:03 GMT
• Title: Branching Time Active Inference: the theory and its generality
• Authors: Th\'eophile Champion, Lancelot Da Costa, Howard Bowman, Marek Grze\'s
• Abstract summary: We present an alternative framework that aims to unify tree search and active inference by casting planning as a structure learning problem. The first propagates the expected free energy forward in time, while the second propagates it backward. Then, we demonstrate that forward and backward propagations are related to active inference and sophisticated inference, respectively, thereby clarifying the differences between those two planning strategies.
• Score: 3.1542695050861544
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Over the last 10 to 15 years, active inference has helped to explain various brain mechanisms from habit formation to dopaminergic discharge and even modelling curiosity. However, the current implementations suffer from an exponential (space and time) complexity class when computing the prior over all the possible policies up to the time-horizon. Fountas et al (2020) used Monte Carlo tree search to address this problem, leading to impressive results in two different tasks. In this paper, we present an alternative framework that aims to unify tree search and active inference by casting planning as a structure learning problem. Two tree search algorithms are then presented. The first propagates the expected free energy forward in time (i.e., towards the leaves), while the second propagates it backward (i.e., towards the root). Then, we demonstrate that forward and backward propagations are related to active inference and sophisticated inference, respectively, thereby clarifying the differences between those two planning strategies.

Related papers

• LiteSearch: Efficacious Tree Search for LLM [70.29796112457662]
  This study introduces a novel guided tree search algorithm with dynamic node selection and node-level exploration budget. Experiments conducted on the GSM8K and TabMWP datasets demonstrate that our approach enjoys significantly lower computational costs compared to baseline methods.
  arXiv  Detail & Related papers  (2024-06-29T05:14:04Z)
• Why do Random Forests Work? Understanding Tree Ensembles as Self-Regularizing Adaptive Smoothers [68.76846801719095]
  We argue that the current high-level dichotomy into bias- and variance-reduction prevalent in statistics is insufficient to understand tree ensembles. We show that forests can improve upon trees by three distinct mechanisms that are usually implicitly entangled.
  arXiv  Detail & Related papers  (2024-02-02T15:36:43Z)
• Probabilistic Tree-of-thought Reasoning for Answering Knowledge-intensive Complex Questions [93.40614719648386]
  Large language models (LLMs) are capable of answering knowledge-intensive complex questions with chain-of-thought (CoT) reasoning. Recent works turn to retrieving external knowledge to augment CoT reasoning. We propose a novel approach: Probabilistic Tree-of-thought Reasoning (ProbTree)
  arXiv  Detail & Related papers  (2023-11-23T12:52:37Z)
• Planning to Learn: A Novel Algorithm for Active Learning during Model-Based Planning [6.3318086812818475]
  We present an extension of SI - sophisticated learning (SL) - that more fully incorporates active learning during planning. SL maintains beliefs about how model parameters would change under the future observations expected under each policy. To accomplish these aims, we make use of a novel, biologically inspired environment designed to highlight the problem structure for which SL offers a unique solution.
  arXiv  Detail & Related papers  (2023-08-15T20:39:23Z)
• Duality Principle and Biologically Plausible Learning: Connecting the Representer Theorem and Hebbian Learning [15.094554860151103]
  We argue that the Representer theorem offers the perfect lens to study biologically plausible learning algorithms. Our work sheds light on the pivotal role of the Representer theorem in advancing our comprehension of neural computation.
  arXiv  Detail & Related papers  (2023-08-02T20:21:18Z)
• RetroGraph: Retrosynthetic Planning with Graph Search [101.92603715499112]
  Retrosynthetic planning aims to find a reaction pathway to synthesize a target molecule. We propose a graph-based search policy that eliminates the redundant explorations of any intermediate molecules. Our method can search a batch of targets together in the graph and remove the inter-target duplication in the tree-based search methods.
  arXiv  Detail & Related papers  (2022-06-23T05:01:29Z)
• Social Interpretable Tree for Pedestrian Trajectory Prediction [75.81745697967608]
  We propose a tree-based method, termed as Social Interpretable Tree (SIT), to address this multi-modal prediction task. A path in the tree from the root to leaf represents an individual possible future trajectory. Despite the hand-crafted tree, the experimental results on ETH-UCY and Stanford Drone datasets demonstrate that our method is capable of matching or exceeding the performance of state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-05-26T12:18:44Z)
• Hybrid Predictive Coding: Inferring, Fast and Slow [62.997667081978825]
  We propose a hybrid predictive coding network that combines both iterative and amortized inference in a principled manner. We demonstrate that our model is inherently sensitive to its uncertainty and adaptively balances balances to obtain accurate beliefs using minimum computational expense.
  arXiv  Detail & Related papers  (2022-04-05T12:52:45Z)
• Branching Time Active Inference: empirical study and complexity class analysis [3.5450828190071655]
  We present an experimental study of the branching-time active inference approach (BTAI) in the context of a maze solving agent. We show that both improved prior preferences and deeper search help mitigate the vulnerability to local minima.
  arXiv  Detail & Related papers  (2021-11-22T15:30:35Z)
• Realising Active Inference in Variational Message Passing: the Outcome-blind Certainty Seeker [3.5450828190071655]
  This paper provides a complete mathematical treatment of the active inference framework -- in discrete time and state spaces. We leverage the theoretical connection between active inference and variational message passing. We show that using a fully factorized variational distribution simplifies the expected free energy.
  arXiv  Detail & Related papers  (2021-04-23T19:40:55Z)
• The Second Type of Uncertainty in Monte Carlo Tree Search [2.564530030795554]
  Monte Carlo Tree Search (MCTS) efficiently balances exploration and exploitation in tree search based on count-derived uncertainty. We first show how, due to the lack of this second uncertainty type, MCTS may completely fail in well-known sparse exploration problems. We then introduce a new algorithm, which estimates the size of the subtree below an action, and leverages this information in the UCB formula to better direct exploration.
  arXiv  Detail & Related papers  (2020-05-19T09:10:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.