Learning Adaptive Language Interfaces through Decomposition

• URL: http://arxiv.org/abs/2010.05190v1
• Date: Sun, 11 Oct 2020 08:27:07 GMT
• Title: Learning Adaptive Language Interfaces through Decomposition
• Authors: Siddharth Karamcheti, Dorsa Sadigh, Percy Liang
• Abstract summary: We introduce a neural semantic parsing system that learns new high-level abstractions through decomposition. Users interactively teach the system by breaking down high-level utterances describing novel behavior into low-level steps.
• Score: 89.21937539950966
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Our goal is to create an interactive natural language interface that efficiently and reliably learns from users to complete tasks in simulated robotics settings. We introduce a neural semantic parsing system that learns new high-level abstractions through decomposition: users interactively teach the system by breaking down high-level utterances describing novel behavior into low-level steps that it can understand. Unfortunately, existing methods either rely on grammars which parse sentences with limited flexibility, or neural sequence-to-sequence models that do not learn efficiently or reliably from individual examples. Our approach bridges this gap, demonstrating the flexibility of modern neural systems, as well as the one-shot reliable generalization of grammar-based methods. Our crowdsourced interactive experiments suggest that over time, users complete complex tasks more efficiently while using our system by leveraging what they just taught. At the same time, getting users to trust the system enough to be incentivized to teach high-level utterances is still an ongoing challenge. We end with a discussion of some of the obstacles we need to overcome to fully realize the potential of the interactive paradigm.

Related papers

• Interpretable Robotic Manipulation from Language [11.207620790833271]
  We introduce an explainable behavior cloning agent, named Ex-PERACT, specifically designed for manipulation tasks. At the top level, the model is tasked with learning a discrete skill code, while at the bottom level, the policy network translates the problem into a voxelized grid and maps the discretized actions to voxel grids. We evaluate our method across eight challenging manipulation tasks utilizing the RLBench benchmark, demonstrating that Ex-PERACT not only achieves competitive policy performance but also effectively bridges the gap between human instructions and machine execution in complex environments.
  arXiv  Detail & Related papers  (2024-05-27T11:02:21Z)
• VAL: Interactive Task Learning with GPT Dialog Parsing [2.6207405455197827]
  Large language models (LLMs) are resistant to brittleness but are not interpretable and cannot learn incrementally. We present VAL, an ITL system with a new philosophy for LLM/symbolic integration. We studied users' interactions with VAL in a video game setting, finding that most users could successfully teach VAL using language they felt was natural.
  arXiv  Detail & Related papers  (2023-10-02T20:45:41Z)
• Language-Driven Representation Learning for Robotics [115.93273609767145]
  Recent work in visual representation learning for robotics demonstrates the viability of learning from large video datasets of humans performing everyday tasks. We introduce a framework for language-driven representation learning from human videos and captions. We find that Voltron's language-driven learning outperform the prior-of-the-art, especially on targeted problems requiring higher-level control.
  arXiv  Detail & Related papers  (2023-02-24T17:29:31Z)
• "No, to the Right" -- Online Language Corrections for Robotic Manipulation via Shared Autonomy [70.45420918526926]
  We present LILAC, a framework for incorporating and adapting to natural language corrections online during execution. Instead of discrete turn-taking between a human and robot, LILAC splits agency between the human and robot. We show that our corrections-aware approach obtains higher task completion rates, and is subjectively preferred by users.
  arXiv  Detail & Related papers  (2023-01-06T15:03:27Z)
• The Whole Truth and Nothing But the Truth: Faithful and Controllable Dialogue Response Generation with Dataflow Transduction and Constrained Decoding [65.34601470417967]
  We describe a hybrid architecture for dialogue response generation that combines the strengths of neural language modeling and rule-based generation. Our experiments show that this system outperforms both rule-based and learned approaches in human evaluations of fluency, relevance, and truthfulness.
  arXiv  Detail & Related papers  (2022-09-16T09:00:49Z)
• Findings from Experiments of On-line Joint Reinforcement Learning of Semantic Parser and Dialogue Manager with real Users [3.9686445409447617]
  On-line learning is pursued in this paper as a convenient way to alleviate these difficulties. A new challenge is to control the cost of the on-line learning borne by the user.
  arXiv  Detail & Related papers  (2021-10-25T18:51:41Z)
• GenNI: Human-AI Collaboration for Data-Backed Text Generation [102.08127062293111]
  Table2Text systems generate textual output based on structured data utilizing machine learning. GenNI (Generation Negotiation Interface) is an interactive visual system for high-level human-AI collaboration in producing descriptive text.
  arXiv  Detail & Related papers  (2021-10-19T18:07:07Z)
• Neural Abstructions: Abstractions that Support Construction for Grounded Language Learning [69.1137074774244]
  Leveraging language interactions effectively requires addressing limitations in the two most common approaches to language grounding. We introduce the idea of neural abstructions: a set of constraints on the inference procedure of a label-conditioned generative model. We show that with this method a user population is able to build a semantic modification for an open-ended house task in Minecraft.
  arXiv  Detail & Related papers  (2021-07-20T07:01:15Z)
• Interactive Teaching for Conversational AI [2.5259192787433706]
  Current conversational AI systems aim to understand a set of pre-designed requests and execute related actions. Motivated by how children learn their first language interacting with adults, this paper describes a new Teachable AI system. It is capable of learning new language nuggets called concepts, directly from end users using live interactive teaching sessions.
  arXiv  Detail & Related papers  (2020-12-02T04:08:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.