Ask & Explore: Grounded Question Answering for Curiosity-Driven Exploration

• URL: http://arxiv.org/abs/2104.11902v1
• Date: Sat, 24 Apr 2021 07:56:31 GMT
• Title: Ask & Explore: Grounded Question Answering for Curiosity-Driven Exploration
• Authors: Jivat Neet Kaur, Yiding Jiang, Paul Pu Liang
• Abstract summary: In many real-world scenarios where extrinsic rewards to the agent are extremely sparse, curiosity has emerged as a useful concept. In this paper, we formulate curiosity based on grounded question answering by encouraging the agent to ask questions about the environment. We show that natural language questions encourage the agent to uncover specific knowledge about their environment such as the physical properties of objects as well as their spatial relationships with other objects.
• Score: 17.28353205476766
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In many real-world scenarios where extrinsic rewards to the agent are extremely sparse, curiosity has emerged as a useful concept providing intrinsic rewards that enable the agent to explore its environment and acquire information to achieve its goals. Despite their strong performance on many sparse-reward tasks, existing curiosity approaches rely on an overly holistic view of state transitions, and do not allow for a structured understanding of specific aspects of the environment. In this paper, we formulate curiosity based on grounded question answering by encouraging the agent to ask questions about the environment and be curious when the answers to these questions change. We show that natural language questions encourage the agent to uncover specific knowledge about their environment such as the physical properties of objects as well as their spatial relationships with other objects, which serve as valuable curiosity rewards to solve sparse-reward tasks more efficiently.

Related papers

• Answerability Fields: Answerable Location Estimation via Diffusion Models [9.234108543963568]
  We propose Answerability Fields, a novel approach to predicting answerability within complex indoor environments. Our results showcase the efficacy of Answerability Fields in guiding scene-understanding tasks.
  arXiv  Detail & Related papers  (2024-07-26T04:02:46Z)
• Analyzing Human Questioning Behavior and Causal Curiosity through Natural Queries [91.70689724416698]
  We present NatQuest, a collection of 13,500 naturally occurring questions from three diverse sources. Our analysis reveals a significant presence of causal questions (up to 42%) within the dataset.
  arXiv  Detail & Related papers  (2024-05-30T17:55:28Z)
• Object Detectors in the Open Environment: Challenges, Solutions, and Outlook [95.3317059617271]
  The dynamic and intricate nature of the open environment poses novel and formidable challenges to object detectors. This paper aims to conduct a comprehensive review and analysis of object detectors in open environments. We propose a framework that includes four quadrants (i.e., out-of-domain, out-of-category, robust learning, and incremental learning) based on the dimensions of the data / target changes.
  arXiv  Detail & Related papers  (2024-03-24T19:32:39Z)
• Information Content Exploration [1.7034813545878589]
  We propose a new intrinsic reward that systemically quantifies exploratory behavior and promotes state coverage. We show that our information theoretic reward induces efficient exploration and outperforms in various games.
  arXiv  Detail & Related papers  (2023-10-10T16:51:32Z)
• A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning [76.47138162283714]
  Forgetting refers to the loss or deterioration of previously acquired information or knowledge. Forgetting is a prevalent phenomenon observed in various other research domains within deep learning. Survey argues that forgetting is a double-edged sword and can be beneficial and desirable in certain cases.
  arXiv  Detail & Related papers  (2023-07-16T16:27:58Z)
• Embodied Agents for Efficient Exploration and Smart Scene Description [47.82947878753809]
  We tackle a setting for visual navigation in which an autonomous agent needs to explore and map an unseen indoor environment. We propose and evaluate an approach that combines recent advances in visual robotic exploration and image captioning on images. Our approach can generate smart scene descriptions that maximize semantic knowledge of the environment and avoid repetitions.
  arXiv  Detail & Related papers  (2023-01-17T19:28:01Z)
• Five Properties of Specific Curiosity You Didn't Know Curious Machines Should Have [4.266866385061999]
  We conduct a comprehensive, multidisciplinary survey of the field of animal and machine curiosity. We introduce and define what we consider to be five of the most important properties of specific curiosity. We show how these properties may be implemented together in a proof-of-concept reinforcement learning agent.
  arXiv  Detail & Related papers  (2022-12-01T00:18:56Z)
• Prototyping three key properties of specific curiosity in computational reinforcement learning [3.1498833540989413]
  We introduce three of the most immediate of these properties and show how they may be implemented together in a proof-of-concept reinforcement learning agent. As we would hope, the agent exhibits short-term directed behaviour while updating long-term preferences to adaptively seek out curiosity-inducing situations. This work presents a novel view into how specific curiosity operates and in the future might be integrated into the behaviour of goal-seeking, decision-making agents in complex environments.
  arXiv  Detail & Related papers  (2022-05-20T18:58:18Z)
• Asking for Knowledge: Training RL Agents to Query External Knowledge Using Language [121.56329458876655]
  We introduce two new environments: the grid-world-based Q-BabyAI and the text-based Q-TextWorld. We propose the "Asking for Knowledge" (AFK) agent, which learns to generate language commands to query for meaningful knowledge.
  arXiv  Detail & Related papers  (2022-05-12T14:20:31Z)
• Information is Power: Intrinsic Control via Information Capture [110.3143711650806]
  We argue that a compact and general learning objective is to minimize the entropy of the agent's state visitation estimated using a latent state-space model. This objective induces an agent to both gather information about its environment, corresponding to reducing uncertainty, and to gain control over its environment, corresponding to reducing the unpredictability of future world states.
  arXiv  Detail & Related papers  (2021-12-07T18:50:42Z)
• Is Curiosity All You Need? On the Utility of Emergent Behaviours from Curious Exploration [20.38772636693469]
  We argue that merely using curiosity for fast environment exploration or as a bonus reward for a specific task does not harness the full potential of this technique. We propose to shift the focus towards retaining the behaviours which emerge during curiosity-based learning.
  arXiv  Detail & Related papers  (2021-09-17T15:28:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.