Overcoming Referential Ambiguity in Language-Guided Goal-Conditioned Reinforcement Learning

• URL: http://arxiv.org/abs/2209.12758v2
• Date: Wed, 27 Sep 2023 07:52:54 GMT
• Title: Overcoming Referential Ambiguity in Language-Guided Goal-Conditioned Reinforcement Learning
• Authors: Hugo Caselles-Dupr\'e, Olivier Sigaud, Mohamed Chetouani
• Abstract summary: The learner can misunderstand the teacher's intentions if the instruction ambiguously refer to features of the object. We study how two concepts derived from cognitive sciences can help resolve those referential ambiguities. We apply those ideas to a teacher/learner setup with two artificial agents on a simulated robotic task.
• Score: 8.715518445626826
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Teaching an agent to perform new tasks using natural language can easily be hindered by ambiguities in interpretation. When a teacher provides an instruction to a learner about an object by referring to its features, the learner can misunderstand the teacher's intentions, for instance if the instruction ambiguously refer to features of the object, a phenomenon called referential ambiguity. We study how two concepts derived from cognitive sciences can help resolve those referential ambiguities: pedagogy (selecting the right instructions) and pragmatism (learning the preferences of the other agents using inductive reasoning). We apply those ideas to a teacher/learner setup with two artificial agents on a simulated robotic task (block-stacking). We show that these concepts improve sample efficiency for training the learner.

Related papers

• Situated Instruction Following [87.37244711380411]
  We propose situated instruction following, which embraces the inherent underspecification and ambiguity of real-world communication. The meaning of situated instructions naturally unfold through the past actions and the expected future behaviors of the human involved. Our experiments indicate that state-of-the-art Embodied Instruction Following (EIF) models lack holistic understanding of situated human intention.
  arXiv  Detail & Related papers  (2024-07-15T19:32:30Z)
• An Incomplete Loop: Deductive, Inductive, and Abductive Learning in Large Language Models [99.31449616860291]
  Modern language models (LMs) can learn to perform new tasks in different ways. In instruction following, the target task is described explicitly in natural language; in few-shot prompting, the task is specified implicitly. In instruction inference, LMs are presented with in-context examples and are then prompted to generate a natural language task description.
  arXiv  Detail & Related papers  (2024-04-03T19:31:56Z)
• LiFT: Unsupervised Reinforcement Learning with Foundation Models as Teachers [59.69716962256727]
  We propose a framework that guides a reinforcement learning agent to acquire semantically meaningful behavior without human feedback. In our framework, the agent receives task instructions grounded in a training environment from large language models. We demonstrate that our method can learn semantically meaningful skills in a challenging open-ended MineDojo environment.
  arXiv  Detail & Related papers  (2023-12-14T14:07:41Z)
• Teacher Perception of Automatically Extracted Grammar Concepts for L2 Language Learning [66.79173000135717]
  We apply this work to teaching two Indian languages, Kannada and Marathi, which do not have well-developed resources for second language learning. We extract descriptions from a natural text corpus that answer questions about morphosyntax (learning of word order, agreement, case marking, or word formation) and semantics (learning of vocabulary). We enlist the help of language educators from schools in North America to perform a manual evaluation, who find the materials have potential to be used for their lesson preparation and learner evaluation.
  arXiv  Detail & Related papers  (2023-10-27T18:17:29Z)
• We're Afraid Language Models Aren't Modeling Ambiguity [136.8068419824318]
  Managing ambiguity is a key part of human language understanding. We characterize ambiguity in a sentence by its effect on entailment relations with another sentence. We show that a multilabel NLI model can flag political claims in the wild that are misleading due to ambiguity.
  arXiv  Detail & Related papers  (2023-04-27T17:57:58Z)
• Language-Conditioned Reinforcement Learning to Solve Misunderstandings with Action Corrections [0.0]
  We present a first formalization and experimental validation of incremental action-repair for robotic instruction-following based on reinforcement learning. We show that a reinforcement learning agent can successfully learn to understand incremental corrections of misunderstood instructions.
  arXiv  Detail & Related papers  (2022-11-18T11:29:04Z)
• Teacher Perception of Automatically Extracted Grammar Concepts for L2 Language Learning [91.49622922938681]
  We present an automatic framework that automatically discovers and visualizing descriptions of different aspects of grammar. Specifically, we extract descriptions from a natural text corpus that answer questions about morphosyntax and semantics. We apply this method for teaching the Indian languages, Kannada and Marathi, which, unlike English, do not have well-developed pedagogical resources.
  arXiv  Detail & Related papers  (2022-06-10T14:52:22Z)
• Grounding Hindsight Instructions in Multi-Goal Reinforcement Learning for Robotics [14.863872352905629]
  This paper focuses on robotic reinforcement learning with sparse rewards for natural language goal representations. We first present a mechanism for hindsight instruction replay utilizing expert feedback. Second, we propose a seq2seq model to generate linguistic hindsight instructions.
  arXiv  Detail & Related papers  (2022-04-08T22:01:36Z)
• Pedagogical Demonstrations and Pragmatic Learning in Artificial Tutor-Learner Interactions [8.715518445626826]
  In this paper, we investigate the implementation of such mechanisms in a tutor-learner setup where both participants are artificial agents in an environment with multiple goals. Using pedagogy from the tutor and pragmatism from the learner, we show substantial improvements over standard learning from demonstrations.
  arXiv  Detail & Related papers  (2022-02-28T21:57:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.