Learning in Hybrid Active Inference Models

• URL: http://arxiv.org/abs/2409.01066v1
• Date: Mon, 2 Sep 2024 08:41:45 GMT
• Title: Learning in Hybrid Active Inference Models
• Authors: Poppy Collis, Ryan Singh, Paul F Kinghorn, Christopher L Buckley,
• Abstract summary: We present a novel hierarchical hybrid active inference agent in which a high-level discrete active inference planner sits above a low-level continuous active inference controller. We make use of recent work in recurrent switching linear dynamical systems which implement end-to-end learning of meaningful discrete representations. We apply our model to the sparse Continuous Mountain Car task, demonstrating fast system identification via enhanced exploration and successful planning.
• Score: 0.8749675983608172
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: An open problem in artificial intelligence is how systems can flexibly learn discrete abstractions that are useful for solving inherently continuous problems. Previous work in computational neuroscience has considered this functional integration of discrete and continuous variables during decision-making under the formalism of active inference (Parr, Friston & de Vries, 2017; Parr & Friston, 2018). However, their focus is on the expressive physical implementation of categorical decisions and the hierarchical mixed generative model is assumed to be known. As a consequence, it is unclear how this framework might be extended to learning. We therefore present a novel hierarchical hybrid active inference agent in which a high-level discrete active inference planner sits above a low-level continuous active inference controller. We make use of recent work in recurrent switching linear dynamical systems (rSLDS) which implement end-to-end learning of meaningful discrete representations via the piecewise linear decomposition of complex continuous dynamics (Linderman et al., 2016). The representations learned by the rSLDS inform the structure of the hybrid decision-making agent and allow us to (1) specify temporally-abstracted sub-goals in a method reminiscent of the options framework, (2) lift the exploration into discrete space allowing us to exploit information-theoretic exploration bonuses and (3) `cache' the approximate solutions to low-level problems in the discrete planner. We apply our model to the sparse Continuous Mountain Car task, demonstrating fast system identification via enhanced exploration and successful planning through the delineation of abstract sub-goals.

Related papers

• Synthesizing Evolving Symbolic Representations for Autonomous Systems [2.4233709516962785]
  This paper presents an open-ended learning system able to synthesize from scratch its experience into a PPDDL representation and update it over time. The system explores the environment and iteratively: (a) discover options, (b) explore the environment using options, (c) abstract the knowledge collected and (d) plan.
  arXiv  Detail & Related papers  (2024-09-18T07:23:26Z)
• Hybrid Recurrent Models Support Emergent Descriptions for Hierarchical Planning and Control [0.8749675983608172]
  A class of hybrid state-space model known as recurrent switching linear dynamical systems (rSLDS) discovers meaningful behavioural units. We propose that the rich representations formed by an rSLDS can provide useful abstractions for planning and control. We present a novel hierarchical model-based algorithm inspired by Active Inference in which a discrete MDP sits above a low-level linear-quadratic controller.
  arXiv  Detail & Related papers  (2024-08-20T16:02:54Z)
• Mamba-FSCIL: Dynamic Adaptation with Selective State Space Model for Few-Shot Class-Incremental Learning [113.89327264634984]
  Few-shot class-incremental learning (FSCIL) confronts the challenge of integrating new classes into a model with minimal training samples. Traditional methods widely adopt static adaptation relying on a fixed parameter space to learn from data that arrive sequentially. We propose a dual selective SSM projector that dynamically adjusts the projection parameters based on the intermediate features for dynamic adaptation.
  arXiv  Detail & Related papers  (2024-07-08T17:09:39Z)
• Distal Interference: Exploring the Limits of Model-Based Continual Learning [0.0]
  Continual learning is hindered by catastrophic interference or forgetting. Despite their practical success, artificial neural networks (ANNs) are prone to catastrophic interference. It is conjectured that continual learning with complexity models requires augmentation of the training data or algorithm.
  arXiv  Detail & Related papers  (2024-02-13T07:07:37Z)
• Entropy-Regularized Token-Level Policy Optimization for Language Agent Reinforcement [67.1393112206885]
  Large Language Models (LLMs) have shown promise as intelligent agents in interactive decision-making tasks. We introduce Entropy-Regularized Token-level Policy Optimization (ETPO), an entropy-augmented RL method tailored for optimizing LLMs at the token level. We assess the effectiveness of ETPO within a simulated environment that models data science code generation as a series of multi-step interactive tasks.
  arXiv  Detail & Related papers  (2024-02-09T07:45:26Z)
• Class-Incremental Mixture of Gaussians for Deep Continual Learning [15.49323098362628]
  We propose end-to-end incorporation of the mixture of Gaussians model into the continual learning framework. We show that our model can effectively learn in memory-free scenarios with fixed extractors.
  arXiv  Detail & Related papers  (2023-07-09T04:33:19Z)
• Safe Multi-agent Learning via Trapping Regions [89.24858306636816]
  We apply the concept of trapping regions, known from qualitative theory of dynamical systems, to create safety sets in the joint strategy space for decentralized learning. We propose a binary partitioning algorithm for verification that candidate sets form trapping regions in systems with known learning dynamics, and a sampling algorithm for scenarios where learning dynamics are not known.
  arXiv  Detail & Related papers  (2023-02-27T14:47:52Z)
• Decomposed Linear Dynamical Systems (dLDS) for learning the latent components of neural dynamics [6.829711787905569]
  We propose a new decomposed dynamical system model that represents complex non-stationary and nonlinear dynamics of time series data. Our model is trained through a dictionary learning procedure, where we leverage recent results in tracking sparse vectors over time. In both continuous-time and discrete-time instructional examples we demonstrate that our model can well approximate the original system.
  arXiv  Detail & Related papers  (2022-06-07T02:25:38Z)
• Weakly Supervised Semantic Segmentation via Alternative Self-Dual Teaching [82.71578668091914]
  This paper establishes a compact learning framework that embeds the classification and mask-refinement components into a unified deep model. We propose a novel alternative self-dual teaching (ASDT) mechanism to encourage high-quality knowledge interaction.
  arXiv  Detail & Related papers  (2021-12-17T11:56:56Z)
• Unsupervised Discriminative Embedding for Sub-Action Learning in Complex Activities [54.615003524001686]
  This paper proposes a novel approach for unsupervised sub-action learning in complex activities. The proposed method maps both visual and temporal representations to a latent space where the sub-actions are learnt discriminatively. We show that the proposed combination of visual-temporal embedding and discriminative latent concepts allow to learn robust action representations in an unsupervised setting.
  arXiv  Detail & Related papers  (2021-04-30T20:07:27Z)
• A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning [48.87397222244402]
  We propose an expansion-based approach for task-free continual learning. Our model successfully performs task-free continual learning for both discriminative and generative tasks.
  arXiv  Detail & Related papers  (2020-01-03T02:07:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.