Intrinsic Motivation in Dynamical Control Systems

• URL: http://arxiv.org/abs/2301.00005v1
• Date: Thu, 29 Dec 2022 05:20:08 GMT
• Title: Intrinsic Motivation in Dynamical Control Systems
• Authors: Stas Tiomkin, Ilya Nemenman, Daniel Polani, Naftali Tishby
• Abstract summary: We investigate an information-theoretic approach to intrinsic motivation, based on maximizing an agent's empowerment. We show that this approach generalizes previous attempts to formalize intrinsic motivation. This opens the door for designing practical artificial, intrinsically motivated controllers.
• Score: 5.635628182420597
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Biological systems often choose actions without an explicit reward signal, a phenomenon known as intrinsic motivation. The computational principles underlying this behavior remain poorly understood. In this study, we investigate an information-theoretic approach to intrinsic motivation, based on maximizing an agent's empowerment (the mutual information between its past actions and future states). We show that this approach generalizes previous attempts to formalize intrinsic motivation, and we provide a computationally efficient algorithm for computing the necessary quantities. We test our approach on several benchmark control problems, and we explain its success in guiding intrinsically motivated behaviors by relating our information-theoretic control function to fundamental properties of the dynamical system representing the combined agent-environment system. This opens the door for designing practical artificial, intrinsically motivated controllers and for linking animal behaviors to their dynamical properties.

Related papers

• Evolving Neural Networks Reveal Emergent Collective Behavior from Minimal Agent Interactions [0.0]
  We investigate how neural networks evolve to control agents' behavior in a dynamic environment. Simpler behaviors, such as lane formation and laminar flow, are characterized by more linear network operations. Specific environmental parameters, such as moderate noise, broader field of view, and lower agent density, promote the evolution of non-linear networks.
  arXiv  Detail & Related papers  (2024-10-25T17:43:00Z)
• Behavior-Inspired Neural Networks for Relational Inference [3.7219180084857473]
  Recent works learn to categorize relationships between agents based on observations of their physical behavior. We introduce a level of abstraction between the observable behavior of agents and the latent categories that determine their behavior. We integrate the physical proximity of agents and their preferences in a nonlinear opinion dynamics model which provides a mechanism to identify mutually exclusive latent categories, predict an agent's evolution in time, and control an agent's physical behavior.
  arXiv  Detail & Related papers  (2024-06-20T21:36:54Z)
• Learning Goal-based Movement via Motivational-based Models in Cognitive Mobile Robots [58.720142291102135]
  Humans have needs motivating their behavior according to intensity and context. We also create preferences associated with each action's perceived pleasure, which is susceptible to changes over time. This makes decision-making more complex, requiring learning to balance needs and preferences according to the context.
  arXiv  Detail & Related papers  (2023-02-20T04:52:24Z)
• Intrinsic Motivation in Model-based Reinforcement Learning: A Brief Review [77.34726150561087]
  This review considers the existing methods for determining intrinsic motivation based on the world model obtained by the agent. The proposed unified framework describes the architecture of agents using a world model and intrinsic motivation to improve learning.
  arXiv  Detail & Related papers  (2023-01-24T15:13:02Z)
• Inference of Affordances and Active Motor Control in Simulated Agents [0.5161531917413706]
  We introduce an output-probabilistic, temporally predictive, modular artificial neural network architecture. We show that our architecture develops latent states that can be interpreted as affordance maps. In combination with active inference, we show that flexible, goal-directed behavior can be invoked.
  arXiv  Detail & Related papers  (2022-02-23T14:13:04Z)
• Information is Power: Intrinsic Control via Information Capture [110.3143711650806]
  We argue that a compact and general learning objective is to minimize the entropy of the agent's state visitation estimated using a latent state-space model. This objective induces an agent to both gather information about its environment, corresponding to reducing uncertainty, and to gain control over its environment, corresponding to reducing the unpredictability of future world states.
  arXiv  Detail & Related papers  (2021-12-07T18:50:42Z)
• Backprop-Free Reinforcement Learning with Active Neural Generative Coding [84.11376568625353]
  We propose a computational framework for learning action-driven generative models without backpropagation of errors (backprop) in dynamic environments. We develop an intelligent agent that operates even with sparse rewards, drawing inspiration from the cognitive theory of planning as inference. The robust performance of our agent offers promising evidence that a backprop-free approach for neural inference and learning can drive goal-directed behavior.
  arXiv  Detail & Related papers  (2021-07-10T19:02:27Z)
• Generative Adversarial Reward Learning for Generalized Behavior Tendency Inference [71.11416263370823]
  We propose a generative inverse reinforcement learning for user behavioral preference modelling. Our model can automatically learn the rewards from user's actions based on discriminative actor-critic network and Wasserstein GAN.
  arXiv  Detail & Related papers  (2021-05-03T13:14:25Z)
• First and Second Order Dynamics in a Hierarchical SOM system for Action Recognition [0.0]
  We present a novel action recognition system that employs a hierarchy of Self-Organizing Maps together with a custom supervised neural network that learns to categorize actions. The system preprocesses the input from a Kinect like 3D camera to exploit the information not only about joint positions, but also their first and second order dynamics.
  arXiv  Detail & Related papers  (2021-04-13T09:46:40Z)
• Tracking Emotions: Intrinsic Motivation Grounded on Multi-Level Prediction Error Dynamics [68.8204255655161]
  We discuss how emotions arise when differences between expected and actual rates of progress towards a goal are experienced. We present an intrinsic motivation architecture that generates behaviors towards self-generated and dynamic goals.
  arXiv  Detail & Related papers  (2020-07-29T06:53:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.