Prototyping three key properties of specific curiosity in computational reinforcement learning

• URL: http://arxiv.org/abs/2205.10407v1
• Date: Fri, 20 May 2022 18:58:18 GMT
• Title: Prototyping three key properties of specific curiosity in computational reinforcement learning
• Authors: Nadia M. Ady, Roshan Shariff, Johannes G\"unther and Patrick M. Pilarski (University of Alberta Department of Computing Science and Alberta Machine Intelligence Institute)
• Abstract summary: We introduce three of the most immediate of these properties and show how they may be implemented together in a proof-of-concept reinforcement learning agent. As we would hope, the agent exhibits short-term directed behaviour while updating long-term preferences to adaptively seek out curiosity-inducing situations. This work presents a novel view into how specific curiosity operates and in the future might be integrated into the behaviour of goal-seeking, decision-making agents in complex environments.
• Score: 3.1498833540989413
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Curiosity for machine agents has been a focus of intense research. The study of human and animal curiosity, particularly specific curiosity, has unearthed several properties that would offer important benefits for machine learners, but that have not yet been well-explored in machine intelligence. In this work, we introduce three of the most immediate of these properties -- directedness, cessation when satisfied, and voluntary exposure -- and show how they may be implemented together in a proof-of-concept reinforcement learning agent; further, we demonstrate how the properties manifest in the behaviour of this agent in a simple non-episodic grid-world environment that includes curiosity-inducing locations and induced targets of curiosity. As we would hope, the agent exhibits short-term directed behaviour while updating long-term preferences to adaptively seek out curiosity-inducing situations. This work therefore presents a novel view into how specific curiosity operates and in the future might be integrated into the behaviour of goal-seeking, decision-making agents in complex environments.

Related papers

• A Comprehensive Review of Few-shot Action Recognition [64.47305887411275]
  Few-shot action recognition aims to address the high cost and impracticality of manually labeling complex and variable video data. It requires accurately classifying human actions in videos using only a few labeled examples per class.
  arXiv  Detail & Related papers  (2024-07-20T03:53:32Z)
• A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning [76.47138162283714]
  Forgetting refers to the loss or deterioration of previously acquired information or knowledge. Forgetting is a prevalent phenomenon observed in various other research domains within deep learning. Survey argues that forgetting is a double-edged sword and can be beneficial and desirable in certain cases.
  arXiv  Detail & Related papers  (2023-07-16T16:27:58Z)
• Five Properties of Specific Curiosity You Didn't Know Curious Machines Should Have [4.266866385061999]
  We conduct a comprehensive, multidisciplinary survey of the field of animal and machine curiosity. We introduce and define what we consider to be five of the most important properties of specific curiosity. We show how these properties may be implemented together in a proof-of-concept reinforcement learning agent.
  arXiv  Detail & Related papers  (2022-12-01T00:18:56Z)
• From Psychological Curiosity to Artificial Curiosity: Curiosity-Driven Learning in Artificial Intelligence Tasks [56.20123080771364]
  Psychological curiosity plays a significant role in human intelligence to enhance learning through exploration and information acquisition. In the Artificial Intelligence (AI) community, artificial curiosity provides a natural intrinsic motivation for efficient learning. CDL has become increasingly popular, where agents are self-motivated to learn novel knowledge.
  arXiv  Detail & Related papers  (2022-01-20T17:07:03Z)
• Information is Power: Intrinsic Control via Information Capture [110.3143711650806]
  We argue that a compact and general learning objective is to minimize the entropy of the agent's state visitation estimated using a latent state-space model. This objective induces an agent to both gather information about its environment, corresponding to reducing uncertainty, and to gain control over its environment, corresponding to reducing the unpredictability of future world states.
  arXiv  Detail & Related papers  (2021-12-07T18:50:42Z)
• Is Curiosity All You Need? On the Utility of Emergent Behaviours from Curious Exploration [20.38772636693469]
  We argue that merely using curiosity for fast environment exploration or as a bonus reward for a specific task does not harness the full potential of this technique. We propose to shift the focus towards retaining the behaviours which emerge during curiosity-based learning.
  arXiv  Detail & Related papers  (2021-09-17T15:28:25Z)
• Ask & Explore: Grounded Question Answering for Curiosity-Driven Exploration [17.28353205476766]
  In many real-world scenarios where extrinsic rewards to the agent are extremely sparse, curiosity has emerged as a useful concept. In this paper, we formulate curiosity based on grounded question answering by encouraging the agent to ask questions about the environment. We show that natural language questions encourage the agent to uncover specific knowledge about their environment such as the physical properties of objects as well as their spatial relationships with other objects.
  arXiv  Detail & Related papers  (2021-04-24T07:56:31Z)
• Towards Open World Object Detection [68.79678648726416]
  ORE: Open World Object Detector is based on contrastive clustering and energy based unknown identification. We find that identifying and characterizing unknown instances helps to reduce confusion in an incremental object detection setting.
  arXiv  Detail & Related papers  (2021-03-03T18:58:18Z)
• The growth and form of knowledge networks by kinesthetic curiosity [0.39325957466009187]
  We show how network science, statistical physics, and philosophy can be integrated into an approach that coheres with and expands the psychological of specific-diversive and perceptual-epistemic curiosity. The kinesthetic model of curiosity offers a vibrant counterpart to the deliberative predictions of model-based reinforcement learning.
  arXiv  Detail & Related papers  (2020-06-04T15:30:41Z)
• Planning to Explore via Self-Supervised World Models [120.31359262226758]
  Plan2Explore is a self-supervised reinforcement learning agent. We present a new approach to self-supervised exploration and fast adaptation to new tasks. Without any training supervision or task-specific interaction, Plan2Explore outperforms prior self-supervised exploration methods.
  arXiv  Detail & Related papers  (2020-05-12T17:59:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.