Active Inference Tree Search in Large POMDPs

• URL: http://arxiv.org/abs/2103.13860v5
• Date: Tue, 08 Oct 2024 09:34:01 GMT
• Title: Active Inference Tree Search in Large POMDPs
• Authors: Domenico Maisto, Francesco Gregoretti, Karl Friston, Giovanni Pezzulo,
• Abstract summary: We introduce a novel method to plan in POMDPs--Active Inference Tree Search (AcT) AcT combines the normative character and biological realism of a leading planning theory in neuroscience (Active Inference) and the scalability of tree search methods in AI. Our simulations show that AcT successfully navigates binary trees that are challenging for sampling-based methods, problems that require adaptive exploration, and the large POMDP problem 'RockSample'--in which AcT reproduces state-of-the-art POMDP solutions.
• Score: 0.0
• License:
• Abstract: The ability to plan ahead efficiently is key for both living organisms and artificial systems. Model-based planning and prospection are widely studied in cognitive neuroscience and artificial intelligence (AI), but from different perspectives--and with different desiderata in mind (biological realism versus scalability) that are difficult to reconcile. Here, we introduce a novel method to plan in POMDPs--Active Inference Tree Search (AcT)--that combines the normative character and biological realism of a leading planning theory in neuroscience (Active Inference) and the scalability of tree search methods in AI. This unification enhances both approaches. On the one hand, tree searches enable the biologically grounded, first principle method of active inference to be applied to large-scale problems. On the other hand, active inference provides a principled solution to the exploration-exploitation dilemma, which is often addressed heuristically in tree search methods. Our simulations show that AcT successfully navigates binary trees that are challenging for sampling-based methods, problems that require adaptive exploration, and the large POMDP problem 'RockSample'--in which AcT reproduces state-of-the-art POMDP solutions. Furthermore, we illustrate how AcT can be used to simulate neurophysiological responses (e.g., in the hippocampus and prefrontal cortex) of humans and other animals that solve large planning problems. These numerical analyses show that Active Tree Search is a principled realisation of neuroscientific and AI planning theories, which offer both biological realism and scalability.

Related papers

• Technical Report: Enhancing LLM Reasoning with Reward-guided Tree Search [95.06503095273395]
  o1-like reasoning approach is challenging, and researchers have been making various attempts to advance this open area of research. We present a preliminary exploration into enhancing the reasoning abilities of LLMs through reward-guided tree search algorithms.
  arXiv  Detail & Related papers  (2024-11-18T16:15:17Z)
• Causal Representation Learning from Multimodal Biological Observations [57.00712157758845]
  We aim to develop flexible identification conditions for multimodal data. We establish identifiability guarantees for each latent component, extending the subspace identification results from prior work. Our key theoretical ingredient is the structural sparsity of the causal connections among distinct modalities.
  arXiv  Detail & Related papers  (2024-11-10T16:40:27Z)
• A Review of Artificial Intelligence based Biological-Tree Construction: Priorities, Methods, Applications and Trends [43.12448177569722]
  Biological tree analysis serves as a pivotal tool in uncovering the evolutionary and differentiation relationships among organisms, genes, and cells. Traditional tree inference methods, while foundational in early studies, face increasing limitations in processing the large-scale, complex datasets. Recent advances in deep learning offer promising solutions, providing enhanced data processing and pattern recognition capabilities.
  arXiv  Detail & Related papers  (2024-10-07T08:00:41Z)
• Dynamic planning in hierarchical active inference [0.0]
  We refer to the ability of the human brain to infer and impose motor trajectories related to cognitive decisions. This study focuses on the topic of dynamic planning in active inference.
  arXiv  Detail & Related papers  (2024-02-18T17:32:53Z)
• A Review of Neuroscience-Inspired Machine Learning [58.72729525961739]
  Bio-plausible credit assignment is compatible with practically any learning condition and is energy-efficient. In this paper, we survey several vital algorithms that model bio-plausible rules of credit assignment in artificial neural networks. We conclude by discussing the future challenges that will need to be addressed in order to make such algorithms more useful in practical applications.
  arXiv  Detail & Related papers  (2024-02-16T18:05:09Z)
• Brain-Inspired Machine Intelligence: A Survey of Neurobiologically-Plausible Credit Assignment [65.268245109828]
  We examine algorithms for conducting credit assignment in artificial neural networks that are inspired or motivated by neurobiology. We organize the ever-growing set of brain-inspired learning schemes into six general families and consider these in the context of backpropagation of errors. The results of this review are meant to encourage future developments in neuro-mimetic systems and their constituent learning processes.
  arXiv  Detail & Related papers  (2023-12-01T05:20:57Z)
• Causal machine learning for single-cell genomics [94.28105176231739]
  We discuss the application of machine learning techniques to single-cell genomics and their challenges. We first present the model that underlies most of current causal approaches to single-cell biology. We then identify open problems in the application of causal approaches to single-cell data.
  arXiv  Detail & Related papers  (2023-10-23T13:35:24Z)
• A Neuro-mimetic Realization of the Common Model of Cognition via Hebbian Learning and Free Energy Minimization [55.11642177631929]
  Large neural generative models are capable of synthesizing semantically rich passages of text or producing complex images. We discuss the COGnitive Neural GENerative system, such an architecture that casts the Common Model of Cognition.
  arXiv  Detail & Related papers  (2023-10-14T23:28:48Z)
• Adaptive patch foraging in deep reinforcement learning agents [4.654270325882834]
  We show that machine learning agents can learn to patch forage adaptively in patterns similar to biological foragers. This work suggests that agents interacting in complex environments with ecologically valid pressures arrive at common solutions.
  arXiv  Detail & Related papers  (2022-10-14T20:16:02Z)
• Information theoretic analysis of computational models as a tool to understand the neural basis of behaviors [0.0]
  One of the greatest research challenges of this century is to understand the neural basis for how behavior emerges in brain-body-environment systems. Computational models provide an alternative framework within which one can study model systems. I provide an introduction, a review and discussion to make a case for how information theoretic analysis of computational models is a potent research methodology.
  arXiv  Detail & Related papers  (2021-06-02T02:08:18Z)
• Grounding Artificial Intelligence in the Origins of Human Behavior [0.0]
  Recent advances in Artificial Intelligence (AI) have revived the quest for agents able to acquire an open-ended repertoire of skills. Research in Human Behavioral Ecology (HBE) seeks to understand how the behaviors characterizing human nature can be conceived as adaptive responses to major changes in the structure of our ecological niche. We propose a framework highlighting the role of environmental complexity in open-ended skill acquisition, grounded in major hypotheses from HBE and recent contributions in Reinforcement learning (RL)
  arXiv  Detail & Related papers  (2020-12-15T19:28:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.