Language-Conditioned Reinforcement Learning to Solve Misunderstandings with Action Corrections

• URL: http://arxiv.org/abs/2211.10168v1
• Date: Fri, 18 Nov 2022 11:29:04 GMT
• Title: Language-Conditioned Reinforcement Learning to Solve Misunderstandings with Action Corrections
• Authors: Frank R\"oder and Manfred Eppe
• Abstract summary: We present a first formalization and experimental validation of incremental action-repair for robotic instruction-following based on reinforcement learning. We show that a reinforcement learning agent can successfully learn to understand incremental corrections of misunderstood instructions.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Human-to-human conversation is not just talking and listening. It is an incremental process where participants continually establish a common understanding to rule out misunderstandings. Current language understanding methods for intelligent robots do not consider this. There exist numerous approaches considering non-understandings, but they ignore the incremental process of resolving misunderstandings. In this article, we present a first formalization and experimental validation of incremental action-repair for robotic instruction-following based on reinforcement learning. To evaluate our approach, we propose a collection of benchmark environments for action correction in language-conditioned reinforcement learning, utilizing a synthetic instructor to generate language goals and their corresponding corrections. We show that a reinforcement learning agent can successfully learn to understand incremental corrections of misunderstood instructions.

Related papers

• SECURE: Semantics-aware Embodied Conversation under Unawareness for Lifelong Robot Learning [17.125080112897102]
  This paper addresses a challenging interactive task learning scenario where the robot is unaware of a concept that's key to solving the instructed task. We propose SECURE, an interactive task learning framework designed to solve such problems by fixing a deficient domain model using embodied conversation. Using SECURE, the robot not only learns from the user's corrective feedback when it makes a mistake, but it also learns to make strategic dialogue decisions for revealing useful evidence about novel concepts for solving the instructed task.
  arXiv  Detail & Related papers  (2024-09-26T11:40:07Z)
• No Train but Gain: Language Arithmetic for training-free Language Adapters enhancement [59.37775534633868]
  We introduce a novel method called language arithmetic, which enables training-free post-processing. The effectiveness of the proposed solution is demonstrated on three downstream tasks in a MAD-X-based set of cross-lingual schemes.
  arXiv  Detail & Related papers  (2024-04-24T08:52:40Z)
• A Definition of Continual Reinforcement Learning [69.56273766737527]
  In a standard view of the reinforcement learning problem, an agent's goal is to efficiently identify a policy that maximizes long-term reward. Continual reinforcement learning refers to the setting in which the best agents never stop learning. We formalize the notion of agents that "never stop learning" through a new mathematical language for analyzing and cataloging agents.
  arXiv  Detail & Related papers  (2023-07-20T17:28:01Z)
• "No, to the Right" -- Online Language Corrections for Robotic Manipulation via Shared Autonomy [70.45420918526926]
  We present LILAC, a framework for incorporating and adapting to natural language corrections online during execution. Instead of discrete turn-taking between a human and robot, LILAC splits agency between the human and robot. We show that our corrections-aware approach obtains higher task completion rates, and is subjectively preferred by users.
  arXiv  Detail & Related papers  (2023-01-06T15:03:27Z)
• Overcoming Referential Ambiguity in Language-Guided Goal-Conditioned Reinforcement Learning [8.715518445626826]
  The learner can misunderstand the teacher's intentions if the instruction ambiguously refer to features of the object. We study how two concepts derived from cognitive sciences can help resolve those referential ambiguities. We apply those ideas to a teacher/learner setup with two artificial agents on a simulated robotic task.
  arXiv  Detail & Related papers  (2022-09-26T15:07:59Z)
• Grounding Hindsight Instructions in Multi-Goal Reinforcement Learning for Robotics [14.863872352905629]
  This paper focuses on robotic reinforcement learning with sparse rewards for natural language goal representations. We first present a mechanism for hindsight instruction replay utilizing expert feedback. Second, we propose a seq2seq model to generate linguistic hindsight instructions.
  arXiv  Detail & Related papers  (2022-04-08T22:01:36Z)
• Teachable Reinforcement Learning via Advice Distillation [161.43457947665073]
  We propose a new supervision paradigm for interactive learning based on "teachable" decision-making systems that learn from structured advice provided by an external teacher. We show that agents that learn from advice can acquire new skills with significantly less human supervision than standard reinforcement learning algorithms.
  arXiv  Detail & Related papers  (2022-03-19T03:22:57Z)
• Language-Conditioned Imitation Learning for Robot Manipulation Tasks [39.40937105264774]
  We introduce a method for incorporating unstructured natural language into imitation learning. At training time, the expert can provide demonstrations along with verbal descriptions in order to describe the underlying intent. The training process then interrelates these two modalities to encode the correlations between language, perception, and motion. The resulting language-conditioned visuomotor policies can be conditioned at runtime on new human commands and instructions.
  arXiv  Detail & Related papers  (2020-10-22T21:49:08Z)
• Learning Adaptive Language Interfaces through Decomposition [89.21937539950966]
  We introduce a neural semantic parsing system that learns new high-level abstractions through decomposition. Users interactively teach the system by breaking down high-level utterances describing novel behavior into low-level steps.
  arXiv  Detail & Related papers  (2020-10-11T08:27:07Z)
• Rethinking Supervised Learning and Reinforcement Learning in Task-Oriented Dialogue Systems [58.724629408229205]
  We demonstrate how traditional supervised learning and a simulator-free adversarial learning method can be used to achieve performance comparable to state-of-the-art RL-based methods. Our main goal is not to beat reinforcement learning with supervised learning, but to demonstrate the value of rethinking the role of reinforcement learning and supervised learning in optimizing task-oriented dialogue systems.
  arXiv  Detail & Related papers  (2020-09-21T12:04:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.