Emergence of Goal-Directed Behaviors via Active Inference with Self-Prior

• URL: http://arxiv.org/abs/2504.11075v1
• Date: Tue, 15 Apr 2025 11:16:27 GMT
• Title: Emergence of Goal-Directed Behaviors via Active Inference with Self-Prior
• Authors: Dongmin Kim, Hoshinori Kanazawa, Naoto Yoshida, Yasuo Kuniyoshi,
• Abstract summary: Infants often exhibit goal-directed behaviors, such as reaching for a sensory stimulus, even when no external reward criterion is provided.<n>We propose a novel density model for an agent's own multimodal sensory experiences, called the "self-prior"<n>Our study implements intrinsically motivated behavior shaped by the agent's own sensory experiences, demonstrating the spontaneous emergence of intentional behavior during early development.
• Score: 4.863927022806184
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Infants often exhibit goal-directed behaviors, such as reaching for a sensory stimulus, even when no external reward criterion is provided. These intrinsically motivated behaviors facilitate spontaneous exploration and learning of the body and environment during early developmental stages. Although computational modeling can offer insight into the mechanisms underlying such behaviors, many existing studies on intrinsic motivation focus primarily on how exploration contributes to acquiring external rewards. In this paper, we propose a novel density model for an agent's own multimodal sensory experiences, called the "self-prior," and investigate whether it can autonomously induce goal-directed behavior. Integrated within an active inference framework based on the free energy principle, the self-prior generates behavioral references purely from an intrinsic process that minimizes mismatches between average past sensory experiences and current observations. This mechanism is also analogous to the acquisition and utilization of a body schema through continuous interaction with the environment. We examine this approach in a simulated environment and confirm that the agent spontaneously reaches toward a tactile stimulus. Our study implements intrinsically motivated behavior shaped by the agent's own sensory experiences, demonstrating the spontaneous emergence of intentional behavior during early development.

Related papers

• Wanting to be Understood [7.40601112616244]
  This paper explores an intrinsic motivation for mutual awareness, hypothesizing that humans possess a fundamental drive to understand.<n>Through simulations of the perceptual crossing paradigm, we explore the effect of various internal reward functions in reinforcement learning agents.<n>Results indicate that while artificial curiosity alone does not lead to a preference for social interaction, rewards emphasizing reciprocal understanding successfully drive agents to prioritize interaction.
  arXiv  Detail & Related papers  (2025-04-09T06:15:24Z)
• Active Inference and Intentional Behaviour [40.19132448481507]
  Recent advances in theoretical biology suggest that basal cognition and sentient behaviour are emergent properties of in vitro cell cultures and neuronal networks. We characterise this kind of self-organisation through the lens of the free energy principle, i.e., as self-evidencing. We investigate these forms of (reactive, sentient, and intentional) behaviour using simulations.
  arXiv  Detail & Related papers  (2023-12-06T09:38:35Z)
• Developmental Curiosity and Social Interaction in Virtual Agents [2.8894038270224858]
  We create a virtual infant agent and place it in a developmentally-inspired 3D environment with no external rewards. We test intrinsic reward functions that are similar to motivations that have been proposed to drive exploration in humans. We find that learning a world model in the presence of an attentive caregiver helps the infant agent learn how to predict scenarios.
  arXiv  Detail & Related papers  (2023-05-22T18:17:07Z)
• Intrinsic Motivation in Dynamical Control Systems [5.635628182420597]
  We investigate an information-theoretic approach to intrinsic motivation, based on maximizing an agent's empowerment. We show that this approach generalizes previous attempts to formalize intrinsic motivation. This opens the door for designing practical artificial, intrinsically motivated controllers.
  arXiv  Detail & Related papers  (2022-12-29T05:20:08Z)
• A Neural Active Inference Model of Perceptual-Motor Learning [62.39667564455059]
  The active inference framework (AIF) is a promising new computational framework grounded in contemporary neuroscience. In this study, we test the ability for the AIF to capture the role of anticipation in the visual guidance of action in humans. We present a novel formulation of the prior function that maps a multi-dimensional world-state to a uni-dimensional distribution of free-energy.
  arXiv  Detail & Related papers  (2022-11-16T20:00:38Z)
• Inference of Affordances and Active Motor Control in Simulated Agents [0.5161531917413706]
  We introduce an output-probabilistic, temporally predictive, modular artificial neural network architecture. We show that our architecture develops latent states that can be interpreted as affordance maps. In combination with active inference, we show that flexible, goal-directed behavior can be invoked.
  arXiv  Detail & Related papers  (2022-02-23T14:13:04Z)
• Information is Power: Intrinsic Control via Information Capture [110.3143711650806]
  We argue that a compact and general learning objective is to minimize the entropy of the agent's state visitation estimated using a latent state-space model. This objective induces an agent to both gather information about its environment, corresponding to reducing uncertainty, and to gain control over its environment, corresponding to reducing the unpredictability of future world states.
  arXiv  Detail & Related papers  (2021-12-07T18:50:42Z)
• Backprop-Free Reinforcement Learning with Active Neural Generative Coding [84.11376568625353]
  We propose a computational framework for learning action-driven generative models without backpropagation of errors (backprop) in dynamic environments. We develop an intelligent agent that operates even with sparse rewards, drawing inspiration from the cognitive theory of planning as inference. The robust performance of our agent offers promising evidence that a backprop-free approach for neural inference and learning can drive goal-directed behavior.
  arXiv  Detail & Related papers  (2021-07-10T19:02:27Z)
• Variational Dynamic for Self-Supervised Exploration in Deep Reinforcement Learning [12.76337275628074]
  In this work, we propose a variational dynamic model based on the conditional variational inference to model the multimodality andgenerativeity. We derive an upper bound of the negative log-likelihood of the environmental transition and use such an upper bound as the intrinsic reward for exploration. Our method outperforms several state-of-the-art environment model-based exploration approaches.
  arXiv  Detail & Related papers  (2020-10-17T09:54:51Z)
• Tracking Emotions: Intrinsic Motivation Grounded on Multi-Level Prediction Error Dynamics [68.8204255655161]
  We discuss how emotions arise when differences between expected and actual rates of progress towards a goal are experienced. We present an intrinsic motivation architecture that generates behaviors towards self-generated and dynamic goals.
  arXiv  Detail & Related papers  (2020-07-29T06:53:13Z)
• Noisy Agents: Self-supervised Exploration by Predicting Auditory Events [127.82594819117753]
  We propose a novel type of intrinsic motivation for Reinforcement Learning (RL) that encourages the agent to understand the causal effect of its actions. We train a neural network to predict the auditory events and use the prediction errors as intrinsic rewards to guide RL exploration. Experimental results on Atari games show that our new intrinsic motivation significantly outperforms several state-of-the-art baselines.
  arXiv  Detail & Related papers  (2020-07-27T17:59:08Z)
• Intrinsic Motivation for Encouraging Synergistic Behavior [55.10275467562764]
  We study the role of intrinsic motivation as an exploration bias for reinforcement learning in sparse-reward synergistic tasks. Our key idea is that a good guiding principle for intrinsic motivation in synergistic tasks is to take actions which affect the world in ways that would not be achieved if the agents were acting on their own.
  arXiv  Detail & Related papers  (2020-02-12T19:34:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.