Natural Language for Human-Robot Collaboration: Problems Beyond Language Grounding

• URL: http://arxiv.org/abs/2110.04441v1
• Date: Sat, 9 Oct 2021 03:24:38 GMT
• Title: Natural Language for Human-Robot Collaboration: Problems Beyond Language Grounding
• Authors: Seth Pate, Wei Xu, Ziyi Yang, Maxwell Love, Siddarth Ganguri, Lawson L.S. Wong
• Abstract summary: We identify several aspects of language processing that are not commonly studied in this context. These include location, planning, and generation. We suggest evaluations for each task, offer baselines for simple methods, and close by discussing challenges and opportunities in studying language for collaboration.
• Score: 10.227242085922613
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: To enable robots to instruct humans in collaborations, we identify several aspects of language processing that are not commonly studied in this context. These include location, planning, and generation. We suggest evaluations for each task, offer baselines for simple methods, and close by discussing challenges and opportunities in studying language for collaboration.

Related papers

• Building a Language-Learning Game for Brazilian Indigenous Languages: A Case of Study [0.0]
  We describe a process to automatically generate language exercises and questions from a dependency treebank and a lexical database for Tupian languages. We conclude that new data gathering processes should be established in partnership with indigenous communities and oriented for educational purposes.
  arXiv  Detail & Related papers  (2024-03-21T16:11:44Z)
• Collecting Interactive Multi-modal Datasets for Grounded Language Understanding [66.30648042100123]
  We formalized the collaborative embodied agent using natural language task. We developed a tool for extensive and scalable data collection. We collected the first dataset for interactive grounded language understanding.
  arXiv  Detail & Related papers  (2022-11-12T02:36:32Z)
• Human-guided Collaborative Problem Solving: A Natural Language based Framework [74.27063862727849]
  Our framework consists of three components -- a natural language engine that parses the language utterances to a formal representation and vice-versa. We illustrate the ability of this framework to address the key challenges of collaborative problem solving by demonstrating it on a collaborative building task in a Minecraft-based blocksworld domain.
  arXiv  Detail & Related papers  (2022-07-19T21:52:37Z)
• IGLU 2022: Interactive Grounded Language Understanding in a Collaborative Environment at NeurIPS 2022 [63.07251290802841]
  We propose IGLU: Interactive Grounded Language Understanding in a Collaborative Environment. The primary goal of the competition is to approach the problem of how to develop interactive embodied agents. This research challenge is naturally related, but not limited, to two fields of study that are highly relevant to the NeurIPS community.
  arXiv  Detail & Related papers  (2022-05-27T06:12:48Z)
• Interactive Grounded Language Understanding in a Collaborative Environment: IGLU 2021 [58.196738777207315]
  We propose emphIGLU: Interactive Grounded Language Understanding in a Collaborative Environment. The primary goal of the competition is to approach the problem of how to build interactive agents that learn to solve a task while provided with grounded natural language instructions in a collaborative environment.
  arXiv  Detail & Related papers  (2022-05-05T01:20:09Z)
• CALVIN: A Benchmark for Language-conditioned Policy Learning for Long-horizon Robot Manipulation Tasks [30.936692970187416]
  General-purpose robots must learn to relate human language to their perceptions and actions. We present CALVIN, an open-source simulated benchmark to learn long-horizon language-conditioned tasks.
  arXiv  Detail & Related papers  (2021-12-06T18:37:33Z)
• NeurIPS 2021 Competition IGLU: Interactive Grounded Language Understanding in a Collaborative Environment [71.11505407453072]
  We propose IGLU: Interactive Grounded Language Understanding in a Collaborative Environment. The primary goal of the competition is to approach the problem of how to build interactive agents that learn to solve a task while provided with grounded natural language instructions in a collaborative environment. This research challenge is naturally related, but not limited, to two fields of study that are highly relevant to the NeurIPS community: Natural Language Understanding and Generation (NLU/G) and Reinforcement Learning (RL)
  arXiv  Detail & Related papers  (2021-10-13T07:13:44Z)
• TEACh: Task-driven Embodied Agents that Chat [14.142543383443032]
  We introduce TEACh, a dataset of over 3,000 human--human, interactive dialogues to complete household tasks in simulation. A Commander with access to oracle information about a task communicates in natural language with a Follower. We propose three benchmarks using TEACh to study embodied intelligence challenges.
  arXiv  Detail & Related papers  (2021-10-01T17:00:14Z)
• Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotation [80.29069988090912]
  We study the problem of learning a range of vision-based manipulation tasks from a large offline dataset of robot interaction. We propose to leverage offline robot datasets with crowd-sourced natural language labels. We find that our approach outperforms both goal-image specifications and language conditioned imitation techniques by more than 25%.
  arXiv  Detail & Related papers  (2021-09-02T17:42:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.