Dialogue with Robots: Proposals for Broadening Participation and Research in the SLIVAR Community

• URL: http://arxiv.org/abs/2404.01158v1
• Date: Mon, 1 Apr 2024 15:03:27 GMT
• Title: Dialogue with Robots: Proposals for Broadening Participation and Research in the SLIVAR Community
• Authors: Casey Kennington, Malihe Alikhani, Heather Pon-Barry, Katherine Atwell, Yonatan Bisk, Daniel Fried, Felix Gervits, Zhao Han, Mert Inan, Michael Johnston, Raj Korpan, Diane Litman, Matthew Marge, Cynthia Matuszek, Ross Mead, Shiwali Mohan, Raymond Mooney, Natalie Parde, Jivko Sinapov, Angela Stewart, Matthew Stone, Stefanie Tellex, Tom Williams,
• Abstract summary: The ability to interact with machines using natural human language is becoming commonplace, but expected. In this paper, we chronicle the recent history of this growing field of spoken dialogue with robots. We offer the community three proposals, the first focused on education, the second on benchmarks, and the third on the modeling of language when it comes to spoken interaction with robots.
• Score: 57.56212633174706
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The ability to interact with machines using natural human language is becoming not just commonplace, but expected. The next step is not just text interfaces, but speech interfaces and not just with computers, but with all machines including robots. In this paper, we chronicle the recent history of this growing field of spoken dialogue with robots and offer the community three proposals, the first focused on education, the second on benchmarks, and the third on the modeling of language when it comes to spoken interaction with robots. The three proposals should act as white papers for any researcher to take and build upon.

Related papers

• Exploring Large Language Models to Facilitate Variable Autonomy for Human-Robot Teaming [4.779196219827508]
  We introduce a novel framework for a GPT-powered multi-robot testbed environment, based on a Unity Virtual Reality (VR) setting. This system allows users to interact with robot agents through natural language, each powered by individual GPT cores. A user study with 12 participants explores the effectiveness of GPT-4 and, more importantly, user strategies when being given the opportunity to converse in natural language within a multi-robot environment.
  arXiv  Detail & Related papers  (2023-12-12T12:26:48Z)
• HandMeThat: Human-Robot Communication in Physical and Social Environments [73.91355172754717]
  HandMeThat is a benchmark for a holistic evaluation of instruction understanding and following in physical and social environments. HandMeThat contains 10,000 episodes of human-robot interactions. We show that both offline and online reinforcement learning algorithms perform poorly on HandMeThat.
  arXiv  Detail & Related papers  (2023-10-05T16:14:46Z)
• WALL-E: Embodied Robotic WAiter Load Lifting with Large Language Model [92.90127398282209]
  This paper investigates the potential of integrating the most recent Large Language Models (LLMs) and existing visual grounding and robotic grasping system. We introduce the WALL-E (Embodied Robotic WAiter load lifting with Large Language model) as an example of this integration. We deploy this LLM-empowered system on the physical robot to provide a more user-friendly interface for the instruction-guided grasping task.
  arXiv  Detail & Related papers  (2023-08-30T11:35:21Z)
• Interactive Conversational Head Generation [68.76774230274076]
  We introduce a new conversation head generation benchmark for synthesizing behaviors of a single interlocutor in a face-to-face conversation. The capability to automatically synthesize interlocutors which can participate in long and multi-turn conversations is vital and offer benefits for various applications.
  arXiv  Detail & Related papers  (2023-07-05T08:06:26Z)
• Evaluating Human-Language Model Interaction [79.33022878034627]
  We develop a new framework, Human-AI Language-based Interaction Evaluation (HALIE), that defines the components of interactive systems. We design five tasks to cover different forms of interaction: social dialogue, question answering, crossword puzzles, summarization, and metaphor generation. We find that better non-interactive performance does not always translate to better human-LM interaction.
  arXiv  Detail & Related papers  (2022-12-19T18:59:45Z)
• Understanding Natural Language in Context [13.112390442564442]
  We focus on cognitive robots, which have some knowledge-based models of the world and operate by reasoning and planning with this model. Our goal in this research is to translate natural language utterances into this robot's formalism. We do so by combining off-the-shelf SOTA language models, planning tools, and the robot's knowledge-base for better communication.
  arXiv  Detail & Related papers  (2022-05-25T11:52:16Z)
• Robotic Speech Synthesis: Perspectives on Interactions, Scenarios, and Ethics [2.6959411243976175]
  We discuss the difficulties of synthesizing non-verbal and interaction-oriented speech signals, particularly backchannels. We present the findings of relevant literature and our prior work, trying to bring the attention of human-robot interaction researchers to design better conversational robots.
  arXiv  Detail & Related papers  (2022-03-17T20:24:17Z)
• Whither the Priors for (Vocal) Interactivity? [6.709659274527638]
  Speech-based communication is often cited as one of the most natural' ways in which humans and robots might interact. Despite this, the resulting interactions are anything but natural' It is argued here that such communication failures are indicative of a deeper malaise.
  arXiv  Detail & Related papers  (2022-03-16T12:06:46Z)
• Self-supervised reinforcement learning for speaker localisation with the iCub humanoid robot [58.2026611111328]
  Looking at a person's face is one of the mechanisms that humans rely on when it comes to filtering speech in noisy environments. Having a robot that can look toward a speaker could benefit ASR performance in challenging environments. We propose a self-supervised reinforcement learning-based framework inspired by the early development of humans.
  arXiv  Detail & Related papers  (2020-11-12T18:02:15Z)
• Spoken Language Interaction with Robots: Research Issues and Recommendations, Report from the NSF Future Directions Workshop [0.819605661841562]
  Meeting human needs requires addressing new challenges in speech technology and user experience design. More powerful adaptation methods are needed, without extensive re-engineering or the collection of massive training data. Since robots operate in real time, their speech and language processing components must also.
  arXiv  Detail & Related papers  (2020-11-11T03:45:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.