Demonstrating the Continual Learning Capabilities and Practical Application of Discrete-Time Active Inference

• URL: http://arxiv.org/abs/2410.00240v1
• Date: Mon, 30 Sep 2024 21:18:46 GMT
• Title: Demonstrating the Continual Learning Capabilities and Practical Application of Discrete-Time Active Inference
• Authors: Rithvik Prakki,
• Abstract summary: Active inference is a mathematical framework for understanding how agents interact with their environments. In this paper, we present a continual learning framework for agents operating in discrete time environments. We demonstrate the agent's ability to relearn and refine its models efficiently, making it suitable for complex domains like finance and healthcare.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Active inference is a mathematical framework for understanding how agents (biological or artificial) interact with their environments, enabling continual adaptation and decision-making. It combines Bayesian inference and free energy minimization to model perception, action, and learning in uncertain and dynamic contexts. Unlike reinforcement learning, active inference integrates exploration and exploitation seamlessly by minimizing expected free energy. In this paper, we present a continual learning framework for agents operating in discrete time environments, using active inference as the foundation. We derive the mathematical formulations of variational and expected free energy and apply them to the design of a self-learning research agent. This agent updates its beliefs and adapts its actions based on new data without manual intervention. Through experiments in changing environments, we demonstrate the agent's ability to relearn and refine its models efficiently, making it suitable for complex domains like finance and healthcare. The paper concludes by discussing how the proposed framework generalizes to other systems, positioning active inference as a flexible approach for adaptive AI.

Related papers

• A Unifying Framework for Action-Conditional Self-Predictive Reinforcement Learning [48.59516337905877]
  Learning a good representation is a crucial challenge for Reinforcement Learning (RL) agents. Recent work has developed theoretical insights into these algorithms. We take a step towards bridging the gap between theory and practice by analyzing an action-conditional self-predictive objective.
  arXiv  Detail & Related papers  (2024-06-04T07:22:12Z)
• Active Inference as a Model of Agency [1.9019250262578857]
  We show that any behaviour complying with physically sound assumptions about how biological agents interact with the world integrates exploration and exploitation. This description, known as active inference, refines the free energy principle, a popular descriptive framework for action and perception originating in neuroscience.
  arXiv  Detail & Related papers  (2024-01-23T17:09:25Z)
• Interactive Natural Language Processing [67.87925315773924]
  Interactive Natural Language Processing (iNLP) has emerged as a novel paradigm within the field of NLP. This paper offers a comprehensive survey of iNLP, starting by proposing a unified definition and framework of the concept.
  arXiv  Detail & Related papers  (2023-05-22T17:18:29Z)
• Homomorphism Autoencoder -- Learning Group Structured Representations from Observed Transitions [51.71245032890532]
  We propose methods enabling an agent acting upon the world to learn internal representations of sensory information consistent with actions that modify it. In contrast to existing work, our approach does not require prior knowledge of the group and does not restrict the set of actions the agent can perform.
  arXiv  Detail & Related papers  (2022-07-25T11:22:48Z)
• Inference of Affordances and Active Motor Control in Simulated Agents [0.5161531917413706]
  We introduce an output-probabilistic, temporally predictive, modular artificial neural network architecture. We show that our architecture develops latent states that can be interpreted as affordance maps. In combination with active inference, we show that flexible, goal-directed behavior can be invoked.
  arXiv  Detail & Related papers  (2022-02-23T14:13:04Z)
• Active Inference in Robotics and Artificial Agents: Survey and Challenges [51.29077770446286]
  We review the state-of-the-art theory and implementations of active inference for state-estimation, control, planning and learning. We showcase relevant experiments that illustrate its potential in terms of adaptation, generalization and robustness.
  arXiv  Detail & Related papers  (2021-12-03T12:10:26Z)
• Backprop-Free Reinforcement Learning with Active Neural Generative Coding [84.11376568625353]
  We propose a computational framework for learning action-driven generative models without backpropagation of errors (backprop) in dynamic environments. We develop an intelligent agent that operates even with sparse rewards, drawing inspiration from the cognitive theory of planning as inference. The robust performance of our agent offers promising evidence that a backprop-free approach for neural inference and learning can drive goal-directed behavior.
  arXiv  Detail & Related papers  (2021-07-10T19:02:27Z)
• Online reinforcement learning with sparse rewards through an active inference capsule [62.997667081978825]
  This paper introduces an active inference agent which minimizes the novel free energy of the expected future. Our model is capable of solving sparse-reward problems with a very high sample efficiency. We also introduce a novel method for approximating the prior model from the reward function, which simplifies the expression of complex objectives.
  arXiv  Detail & Related papers  (2021-06-04T10:03:36Z)
• Feature-Based Interpretable Reinforcement Learning based on State-Transition Models [3.883460584034766]
  Growing concerns regarding the operational usage of AI models in the real-world has caused a surge of interest in explaining AI models' decisions to humans. We propose a method for offering local explanations on risk in reinforcement learning.
  arXiv  Detail & Related papers  (2021-05-14T23:43:11Z)
• Realising Active Inference in Variational Message Passing: the Outcome-blind Certainty Seeker [3.5450828190071655]
  This paper provides a complete mathematical treatment of the active inference framework -- in discrete time and state spaces. We leverage the theoretical connection between active inference and variational message passing. We show that using a fully factorized variational distribution simplifies the expected free energy.
  arXiv  Detail & Related papers  (2021-04-23T19:40:55Z)
• Deep active inference agents using Monte-Carlo methods [3.8233569758620054]
  We present a neural architecture for building deep active inference agents in continuous state-spaces using Monte-Carlo sampling. Our approach enables agents to learn environmental dynamics efficiently, while maintaining task performance. Results show that deep active inference provides a flexible framework to develop biologically-inspired intelligent agents.
  arXiv  Detail & Related papers  (2020-06-07T15:10:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.