SPA: Verbal Interactions between Agents and Avatars in Shared Virtual Environments using Propositional Planning

• URL: http://arxiv.org/abs/2002.03246v1
• Date: Sat, 8 Feb 2020 23:15:06 GMT
• Title: SPA: Verbal Interactions between Agents and Avatars in Shared Virtual Environments using Propositional Planning
• Authors: Andrew Best, Sahil Narang, Dinesh Manocha
• Abstract summary: Sense-Plan-Ask, or SPA, generates plausible verbal interactions between virtual human-like agents and user avatars in shared virtual environments. We find that our algorithm creates a small runtime cost and enables agents to complete their goals more effectively than agents without the ability to leverage natural-language communication.
• Score: 61.335252950832256
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a novel approach for generating plausible verbal interactions between virtual human-like agents and user avatars in shared virtual environments. Sense-Plan-Ask, or SPA, extends prior work in propositional planning and natural language processing to enable agents to plan with uncertain information, and leverage question and answer dialogue with other agents and avatars to obtain the needed information and complete their goals. The agents are additionally able to respond to questions from the avatars and other agents using natural-language enabling real-time multi-agent multi-avatar communication environments. Our algorithm can simulate tens of virtual agents at interactive rates interacting, moving, communicating, planning, and replanning. We find that our algorithm creates a small runtime cost and enables agents to complete their goals more effectively than agents without the ability to leverage natural-language communication. We demonstrate quantitative results on a set of simulated benchmarks and detail the results of a preliminary user-study conducted to evaluate the plausibility of the virtual interactions generated by SPA. Overall, we find that participants prefer SPA to prior techniques in 84\% of responses including significant benefits in terms of the plausibility of natural-language interactions and the positive impact of those interactions.

Related papers

• ReSpAct: Harmonizing Reasoning, Speaking, and Acting Towards Building Large Language Model-Based Conversational AI Agents [11.118991548784459]
  Large language model (LLM)-based agents have been increasingly used to interact with external environments. Current frameworks do not enable these agents to work with users and interact with them to align on the details of their tasks. This work introduces ReSpAct, a novel framework that combines the essential skills for building task-oriented "conversational" agents.
  arXiv  Detail & Related papers  (2024-11-01T15:57:45Z)
• Simulating User Agents for Embodied Conversational-AI [9.402740034754455]
  We build a large language model (LLM)-based user agent that can simulate user behavior during interactions with an embodied agent. We evaluate our user agent's ability to generate human-like behaviors by comparing its simulated dialogues with the TEACh dataset.
  arXiv  Detail & Related papers  (2024-10-31T00:56:08Z)
• Multimodal Fusion with LLMs for Engagement Prediction in Natural Conversation [70.52558242336988]
  We focus on predicting engagement in dyadic interactions by scrutinizing verbal and non-verbal cues, aiming to detect signs of disinterest or confusion. In this work, we collect a dataset featuring 34 participants engaged in casual dyadic conversations, each providing self-reported engagement ratings at the end of each conversation. We introduce a novel fusion strategy using Large Language Models (LLMs) to integrate multiple behavior modalities into a multimodal transcript''
  arXiv  Detail & Related papers  (2024-09-13T18:28:12Z)
• AntEval: Evaluation of Social Interaction Competencies in LLM-Driven Agents [65.16893197330589]
  Large Language Models (LLMs) have demonstrated their ability to replicate human behaviors across a wide range of scenarios. However, their capability in handling complex, multi-character social interactions has yet to be fully explored. We introduce the Multi-Agent Interaction Evaluation Framework (AntEval), encompassing a novel interaction framework and evaluation methods.
  arXiv  Detail & Related papers  (2024-01-12T11:18:00Z)
• Agent AI: Surveying the Horizons of Multimodal Interaction [83.18367129924997]
  "Agent AI" is a class of interactive systems that can perceive visual stimuli, language inputs, and other environmentally-grounded data. We envision a future where people can easily create any virtual reality or simulated scene and interact with agents embodied within the virtual environment.
  arXiv  Detail & Related papers  (2024-01-07T19:11:18Z)
• AgentCF: Collaborative Learning with Autonomous Language Agents for Recommender Systems [112.76941157194544]
  We propose AgentCF for simulating user-item interactions in recommender systems through agent-based collaborative filtering. We creatively consider not only users but also items as agents, and develop a collaborative learning approach that optimize both kinds of agents together. Overall, the optimized agents exhibit diverse interaction behaviors within our framework, including user-item, user-user, item-item, and collective interactions.
  arXiv  Detail & Related papers  (2023-10-13T16:37:14Z)
• Interactive Natural Language Processing [67.87925315773924]
  Interactive Natural Language Processing (iNLP) has emerged as a novel paradigm within the field of NLP. This paper offers a comprehensive survey of iNLP, starting by proposing a unified definition and framework of the concept.
  arXiv  Detail & Related papers  (2023-05-22T17:18:29Z)
• Imitating Interactive Intelligence [24.95842455898523]
  We study how to design artificial agents that can interact naturally with humans using the simplification of a virtual environment. To build agents that can robustly interact with humans, we would ideally train them while they interact with humans. We use ideas from inverse reinforcement learning to reduce the disparities between human-human and agent-agent interactive behaviour.
  arXiv  Detail & Related papers  (2020-12-10T13:55:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.