Five Properties of Specific Curiosity You Didn't Know Curious Machines Should Have

• URL: http://arxiv.org/abs/2212.00187v1
• Date: Thu, 1 Dec 2022 00:18:56 GMT
• Title: Five Properties of Specific Curiosity You Didn't Know Curious Machines Should Have
• Authors: Nadia M. Ady, Roshan Shariff, Johannes G\"unther, Patrick M. Pilarski
• Abstract summary: We conduct a comprehensive, multidisciplinary survey of the field of animal and machine curiosity. We introduce and define what we consider to be five of the most important properties of specific curiosity. We show how these properties may be implemented together in a proof-of-concept reinforcement learning agent.
• Score: 4.266866385061999
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Curiosity for machine agents has been a focus of lively research activity. The study of human and animal curiosity, particularly specific curiosity, has unearthed several properties that would offer important benefits for machine learners, but that have not yet been well-explored in machine intelligence. In this work, we conduct a comprehensive, multidisciplinary survey of the field of animal and machine curiosity. As a principal contribution of this work, we use this survey as a foundation to introduce and define what we consider to be five of the most important properties of specific curiosity: 1) directedness towards inostensible referents, 2) cessation when satisfied, 3) voluntary exposure, 4) transience, and 5) coherent long-term learning. As a second main contribution of this work, we show how these properties may be implemented together in a proof-of-concept reinforcement learning agent: we demonstrate how the properties manifest in the behaviour of this agent in a simple non-episodic grid-world environment that includes curiosity-inducing locations and induced targets of curiosity. As we would hope, our example of a computational specific curiosity agent exhibits short-term directed behaviour while updating long-term preferences to adaptively seek out curiosity-inducing situations. This work, therefore, presents a landmark synthesis and translation of specific curiosity to the domain of machine learning and reinforcement learning and provides a novel view into how specific curiosity operates and in the future might be integrated into the behaviour of goal-seeking, decision-making computational agents in complex environments.

Related papers

• A Comprehensive Review of Few-shot Action Recognition [64.47305887411275]
  Few-shot action recognition aims to address the high cost and impracticality of manually labeling complex and variable video data. It requires accurately classifying human actions in videos using only a few labeled examples per class.
  arXiv  Detail & Related papers  (2024-07-20T03:53:32Z)
• Analyzing Human Questioning Behavior and Causal Curiosity through Natural Queries [91.70689724416698]
  We present NatQuest, a collection of 13,500 naturally occurring questions from three diverse sources. Our analysis reveals a significant presence of causal questions (up to 42%) within the dataset.
  arXiv  Detail & Related papers  (2024-05-30T17:55:28Z)
• A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning [58.107474025048866]
  Forgetting refers to the loss or deterioration of previously acquired knowledge. Forgetting is a prevalent phenomenon observed in various other research domains within deep learning.
  arXiv  Detail & Related papers  (2023-07-16T16:27:58Z)
• Prototyping three key properties of specific curiosity in computational reinforcement learning [3.1498833540989413]
  We introduce three of the most immediate of these properties and show how they may be implemented together in a proof-of-concept reinforcement learning agent. As we would hope, the agent exhibits short-term directed behaviour while updating long-term preferences to adaptively seek out curiosity-inducing situations. This work presents a novel view into how specific curiosity operates and in the future might be integrated into the behaviour of goal-seeking, decision-making agents in complex environments.
  arXiv  Detail & Related papers  (2022-05-20T18:58:18Z)
• A Novel Multimodal Approach for Studying the Dynamics of Curiosity in Small Group Learning [2.55061802822074]
  We propose an integrated socio-cognitive account of curiosity that ties observable behaviors in peers to underlying curiosity states. We make a bipartite distinction between individual and interpersonal functions that contribute to curiosity, and multimodal behaviors that fulfill these functions. This work is a step towards designing learning technologies that can recognize and evoke moment-by-moment curiosity during learning in social contexts.
  arXiv  Detail & Related papers  (2022-04-01T16:12:40Z)
• From Psychological Curiosity to Artificial Curiosity: Curiosity-Driven Learning in Artificial Intelligence Tasks [56.20123080771364]
  Psychological curiosity plays a significant role in human intelligence to enhance learning through exploration and information acquisition. In the Artificial Intelligence (AI) community, artificial curiosity provides a natural intrinsic motivation for efficient learning. CDL has become increasingly popular, where agents are self-motivated to learn novel knowledge.
  arXiv  Detail & Related papers  (2022-01-20T17:07:03Z)
• Information is Power: Intrinsic Control via Information Capture [110.3143711650806]
  We argue that a compact and general learning objective is to minimize the entropy of the agent's state visitation estimated using a latent state-space model. This objective induces an agent to both gather information about its environment, corresponding to reducing uncertainty, and to gain control over its environment, corresponding to reducing the unpredictability of future world states.
  arXiv  Detail & Related papers  (2021-12-07T18:50:42Z)
• Is Curiosity All You Need? On the Utility of Emergent Behaviours from Curious Exploration [20.38772636693469]
  We argue that merely using curiosity for fast environment exploration or as a bonus reward for a specific task does not harness the full potential of this technique. We propose to shift the focus towards retaining the behaviours which emerge during curiosity-based learning.
  arXiv  Detail & Related papers  (2021-09-17T15:28:25Z)
• Ask & Explore: Grounded Question Answering for Curiosity-Driven Exploration [17.28353205476766]
  In many real-world scenarios where extrinsic rewards to the agent are extremely sparse, curiosity has emerged as a useful concept. In this paper, we formulate curiosity based on grounded question answering by encouraging the agent to ask questions about the environment. We show that natural language questions encourage the agent to uncover specific knowledge about their environment such as the physical properties of objects as well as their spatial relationships with other objects.
  arXiv  Detail & Related papers  (2021-04-24T07:56:31Z)
• Towards Open World Object Detection [68.79678648726416]
  ORE: Open World Object Detector is based on contrastive clustering and energy based unknown identification. We find that identifying and characterizing unknown instances helps to reduce confusion in an incremental object detection setting.
  arXiv  Detail & Related papers  (2021-03-03T18:58:18Z)
• The growth and form of knowledge networks by kinesthetic curiosity [0.39325957466009187]
  We show how network science, statistical physics, and philosophy can be integrated into an approach that coheres with and expands the psychological of specific-diversive and perceptual-epistemic curiosity. The kinesthetic model of curiosity offers a vibrant counterpart to the deliberative predictions of model-based reinforcement learning.
  arXiv  Detail & Related papers  (2020-06-04T15:30:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.