Ask Before You Act: Generalising to Novel Environments by Asking Questions

• URL: http://arxiv.org/abs/2209.04665v2
• Date: Tue, 13 Sep 2022 10:24:16 GMT
• Title: Ask Before You Act: Generalising to Novel Environments by Asking Questions
• Authors: Ross Murphy, Sergey Mosesov, Javier Leguina Peral, Thymo ter Doest
• Abstract summary: We investigate the ability of an RL agent to learn to ask natural language questions as a tool to understand its environment. We do this by endowing this agent with the ability of asking "yes-no" questions to an all-knowing Oracle. We observe a significant increase in generalisation performance compared to a baseline agent unable to ask questions.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Solving temporally-extended tasks is a challenge for most reinforcement learning (RL) algorithms [arXiv:1906.07343]. We investigate the ability of an RL agent to learn to ask natural language questions as a tool to understand its environment and achieve greater generalisation performance in novel, temporally-extended environments. We do this by endowing this agent with the ability of asking "yes-no" questions to an all-knowing Oracle. This allows the agent to obtain guidance regarding the task at hand, while limiting the access to new information. To study the emergence of such natural language questions in the context of temporally-extended tasks we first train our agent in a Mini-Grid environment. We then transfer the trained agent to a different, harder environment. We observe a significant increase in generalisation performance compared to a baseline agent unable to ask questions. Through grounding its understanding of natural language in its environment, the agent can reason about the dynamics of its environment to the point that it can ask new, relevant questions when deployed in a novel environment.

Related papers

• A Survey on Complex Tasks for Goal-Directed Interactive Agents [60.53915548970061]
  This survey compiles relevant tasks and environments for evaluating goal-directed interactive agents. An up-to-date compilation of relevant resources can be found on our project website.
  arXiv  Detail & Related papers  (2024-09-27T08:17:53Z)
• Analyzing Human Questioning Behavior and Causal Curiosity through Natural Queries [91.70689724416698]
  We present NatQuest, a collection of 13,500 naturally occurring questions from three diverse sources. Our analysis reveals a significant presence of causal questions (up to 42%) within the dataset.
  arXiv  Detail & Related papers  (2024-05-30T17:55:28Z)
• HAZARD Challenge: Embodied Decision Making in Dynamically Changing Environments [93.94020724735199]
  HAZARD consists of three unexpected disaster scenarios, including fire, flood, and wind. This benchmark enables us to evaluate autonomous agents' decision-making capabilities across various pipelines.
  arXiv  Detail & Related papers  (2024-01-23T18:59:43Z)
• Efficient Open-world Reinforcement Learning via Knowledge Distillation and Autonomous Rule Discovery [5.680463564655267]
  Rule-driven deep Q-learning agent (RDQ) as one possible implementation of framework. We show that RDQ successfully extracts task-specific rules as it interacts with the world. In experiments, we show that the RDQ agent is significantly more resilient to the novelties than the baseline agents.
  arXiv  Detail & Related papers  (2023-11-24T04:12:50Z)
• Keeping the Questions Conversational: Using Structured Representations to Resolve Dependency in Conversational Question Answering [26.997542897342164]
  We propose a novel framework, CONVSR (CONVQA using Structured Representations) for capturing and generating intermediate representations as conversational cues. We test our model on the QuAC and CANARD datasets and illustrate by experimental results that our proposed framework achieves a better F1 score than the standard question rewriting model.
  arXiv  Detail & Related papers  (2023-04-14T13:42:32Z)
• DANLI: Deliberative Agent for Following Natural Language Instructions [9.825482203664963]
  We propose a neuro-symbolic deliberative agent that applies reasoning and planning based on its neural and symbolic representations acquired from past experience. We show that our deliberative agent achieves greater than 70% improvement over reactive baselines on the challenging TEACh benchmark.
  arXiv  Detail & Related papers  (2022-10-22T15:57:01Z)
• Learning to Query Internet Text for Informing Reinforcement Learning Agents [36.69880704465014]
  We tackle the problem of extracting useful information from natural language found in the wild. We train reinforcement learning agents to learn to query these sources as a human would. We show that our method correctly learns to execute queries to maximize reward in a reinforcement learning setting.
  arXiv  Detail & Related papers  (2022-05-25T23:07:10Z)
• Asking for Knowledge: Training RL Agents to Query External Knowledge Using Language [121.56329458876655]
  We introduce two new environments: the grid-world-based Q-BabyAI and the text-based Q-TextWorld. We propose the "Asking for Knowledge" (AFK) agent, which learns to generate language commands to query for meaningful knowledge.
  arXiv  Detail & Related papers  (2022-05-12T14:20:31Z)
• Interactive Grounded Language Understanding in a Collaborative Environment: IGLU 2021 [58.196738777207315]
  We propose emphIGLU: Interactive Grounded Language Understanding in a Collaborative Environment. The primary goal of the competition is to approach the problem of how to build interactive agents that learn to solve a task while provided with grounded natural language instructions in a collaborative environment.
  arXiv  Detail & Related papers  (2022-05-05T01:20:09Z)
• Inquisitive Question Generation for High Level Text Comprehension [60.21497846332531]
  We introduce INQUISITIVE, a dataset of 19K questions that are elicited while a person is reading through a document. We show that readers engage in a series of pragmatic strategies to seek information. We evaluate question generation models based on GPT-2 and show that our model is able to generate reasonable questions.
  arXiv  Detail & Related papers  (2020-10-04T19:03:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.