On Predictive planning and counterfactual learning in active inference

• URL: http://arxiv.org/abs/2403.12417v1
• Date: Tue, 19 Mar 2024 04:02:31 GMT
• Title: On Predictive planning and counterfactual learning in active inference
• Authors: Aswin Paul, Takuya Isomura, Adeel Razi,
• Abstract summary: In this paper, we examine two decision-making schemes in active inference based on 'planning' and 'learning from experience' We introduce a mixed model that navigates the data-complexity trade-off between these strategies. We evaluate our proposed model in a challenging grid-world scenario that requires adaptability from the agent.
• Score: 0.20482269513546453
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Given the rapid advancement of artificial intelligence, understanding the foundations of intelligent behaviour is increasingly important. Active inference, regarded as a general theory of behaviour, offers a principled approach to probing the basis of sophistication in planning and decision-making. In this paper, we examine two decision-making schemes in active inference based on 'planning' and 'learning from experience'. Furthermore, we also introduce a mixed model that navigates the data-complexity trade-off between these strategies, leveraging the strengths of both to facilitate balanced decision-making. We evaluate our proposed model in a challenging grid-world scenario that requires adaptability from the agent. Additionally, our model provides the opportunity to analyze the evolution of various parameters, offering valuable insights and contributing to an explainable framework for intelligent decision-making.

Related papers

• LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Models [75.89014602596673]
  Strategic reasoning requires understanding and predicting adversary actions in multi-agent settings while adjusting strategies accordingly. We explore the scopes, applications, methodologies, and evaluation metrics related to strategic reasoning with Large Language Models. It underscores the importance of strategic reasoning as a critical cognitive capability and offers insights into future research directions and potential improvements.
  arXiv  Detail & Related papers  (2024-04-01T16:50:54Z)
• Explaining by Imitating: Understanding Decisions by Interpretable Policy Learning [72.80902932543474]
  Understanding human behavior from observed data is critical for transparency and accountability in decision-making. Consider real-world settings such as healthcare, in which modeling a decision-maker's policy is challenging. We propose a data-driven representation of decision-making behavior that inheres transparency by design, accommodates partial observability, and operates completely offline.
  arXiv  Detail & Related papers  (2023-10-28T13:06:14Z)
• Generative Intrinsic Optimization: Intrinsic Control with Model Learning [5.439020425819001]
  Future sequence represents the outcome after executing the action into the environment. Explicit outcomes may vary across state, return, or trajectory serving different purposes such as credit assignment or imitation learning. We propose a policy scheme that seamlessly incorporates the mutual information, ensuring convergence to the optimal policy.
  arXiv  Detail & Related papers  (2023-10-12T07:50:37Z)
• On the Complexity of Adversarial Decision Making [101.14158787665252]
  We show that the Decision-Estimation Coefficient is necessary and sufficient to obtain low regret for adversarial decision making. We provide new structural results that connect the Decision-Estimation Coefficient to variants of other well-known complexity measures.
  arXiv  Detail & Related papers  (2022-06-27T06:20:37Z)
• Inverse Online Learning: Understanding Non-Stationary and Reactionary Policies [79.60322329952453]
  We show how to develop interpretable representations of how agents make decisions. By understanding the decision-making processes underlying a set of observed trajectories, we cast the policy inference problem as the inverse to this online learning problem. We introduce a practical algorithm for retrospectively estimating such perceived effects, alongside the process through which agents update them. Through application to the analysis of UNOS organ donation acceptance decisions, we demonstrate that our approach can bring valuable insights into the factors that govern decision processes and how they change over time.
  arXiv  Detail & Related papers  (2022-03-14T17:40:42Z)
• Representations of epistemic uncertainty and awareness in data-driven strategies [0.0]
  We present a theoretical model for uncertainty in knowledge representation and its transfer mediated by agents. We look at inequivalent knowledge representations in terms of inferences, preference relations, and information measures. We discuss some implications of the proposed model for data-driven strategies.
  arXiv  Detail & Related papers  (2021-10-21T21:18:21Z)
• Deceptive Decision-Making Under Uncertainty [25.197098169762356]
  We study the design of autonomous agents that are capable of deceiving outside observers about their intentions while carrying out tasks. By modeling the agent's behavior as a Markov decision process, we consider a setting where the agent aims to reach one of multiple potential goals. We propose a novel approach to model observer predictions based on the principle of maximum entropy and to efficiently generate deceptive strategies.
  arXiv  Detail & Related papers  (2021-09-14T14:56:23Z)
• Forethought and Hindsight in Credit Assignment [62.05690959741223]
  We work to understand the gains and peculiarities of planning employed as forethought via forward models or as hindsight operating with backward models. We investigate the best use of models in planning, primarily focusing on the selection of states in which predictions should be (re)-evaluated.
  arXiv  Detail & Related papers  (2020-10-26T16:00:47Z)
• Model-based Reinforcement Learning: A Survey [2.564530030795554]
  Sequential decision making, commonly formalized as Markov Decision Process (MDP) optimization, is a important challenge in artificial intelligence. Two key approaches to this problem are reinforcement learning (RL) and planning. This paper presents a survey of the integration of both fields, better known as model-based reinforcement learning.
  arXiv  Detail & Related papers  (2020-06-30T12:10:07Z)
• Inverse Active Sensing: Modeling and Understanding Timely Decision-Making [111.07204912245841]
  We develop a framework for the general setting of evidence-based decision-making under endogenous, context-dependent time pressure. We demonstrate how it enables modeling intuitive notions of surprise, suspense, and optimality in decision strategies.
  arXiv  Detail & Related papers  (2020-06-25T02:30:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.