Constrained Human-AI Cooperation: An Inclusive Embodied Social Intelligence Challenge

• URL: http://arxiv.org/abs/2411.01796v2
• Date: Tue, 05 Nov 2024 03:28:38 GMT
• Title: Constrained Human-AI Cooperation: An Inclusive Embodied Social Intelligence Challenge
• Authors: Weihua Du, Qiushi Lyu, Jiaming Shan, Zhenting Qi, Hongxin Zhang, Sunli Chen, Andi Peng, Tianmin Shu, Kwonjoon Lee, Behzad Dariush, Chuang Gan,
• Abstract summary: CHAIC is an inclusive embodied social intelligence challenge designed to test social perception and cooperation in embodied agents. In CHAIC, the goal is for an embodied agent equipped with egocentric observations to assist a human who may be operating under physical constraints. We benchmark planning- and learning-based baselines on the challenge and introduce a new method that leverages large language models and behavior modeling.
• Score: 47.74313897705183
• License:
• Abstract: We introduce Constrained Human-AI Cooperation (CHAIC), an inclusive embodied social intelligence challenge designed to test social perception and cooperation in embodied agents. In CHAIC, the goal is for an embodied agent equipped with egocentric observations to assist a human who may be operating under physical constraints -- e.g., unable to reach high places or confined to a wheelchair -- in performing common household or outdoor tasks as efficiently as possible. To achieve this, a successful helper must: (1) infer the human's intents and constraints by following the human and observing their behaviors (social perception), and (2) make a cooperative plan tailored to the human partner to solve the task as quickly as possible, working together as a team (cooperative planning). To benchmark this challenge, we create four new agents with real physical constraints and eight long-horizon tasks featuring both indoor and outdoor scenes with various constraints, emergency events, and potential risks. We benchmark planning- and learning-based baselines on the challenge and introduce a new method that leverages large language models and behavior modeling. Empirical evaluations demonstrate the effectiveness of our benchmark in enabling systematic assessment of key aspects of machine social intelligence. Our benchmark and code are publicly available at https://github.com/UMass-Foundation-Model/CHAIC.

Related papers

• Online Learning of Human Constraints from Feedback in Shared Autonomy [25.173950581816086]
  Real-time collaboration with humans poses challenges due to the different behavior patterns of humans resulting from diverse physical constraints. We learn a human constraints model that considers the diverse behaviors of different human operators. We propose an augmentative assistant agent capable of learning and adapting to human physical constraints.
  arXiv  Detail & Related papers  (2024-03-05T13:53:48Z)
• Large Language Model-based Human-Agent Collaboration for Complex Task Solving [94.3914058341565]
  We introduce the problem of Large Language Models (LLMs)-based human-agent collaboration for complex task-solving. We propose a Reinforcement Learning-based Human-Agent Collaboration method, ReHAC. This approach includes a policy model designed to determine the most opportune stages for human intervention within the task-solving process.
  arXiv  Detail & Related papers  (2024-02-20T11:03:36Z)
• SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents [107.4138224020773]
  We present SOTOPIA, an open-ended environment to simulate complex social interactions between artificial agents and humans. In our environment, agents role-play and interact under a wide variety of scenarios; they coordinate, collaborate, exchange, and compete with each other to achieve complex social goals. We find that GPT-4 achieves a significantly lower goal completion rate than humans and struggles to exhibit social commonsense reasoning and strategic communication skills.
  arXiv  Detail & Related papers  (2023-10-18T02:27:01Z)
• MetaAgents: Simulating Interactions of Human Behaviors for LLM-based Task-oriented Coordination via Collaborative Generative Agents [27.911816995891726]
  We introduce collaborative generative agents, endowing LLM-based Agents with consistent behavior patterns and task-solving abilities. We propose a novel framework that equips collaborative generative agents with human-like reasoning abilities and specialized skills. Our work provides valuable insights into the role and evolution of Large Language Models in task-oriented social simulations.
  arXiv  Detail & Related papers  (2023-10-10T10:17:58Z)
• Robust Planning for Human-Robot Joint Tasks with Explicit Reasoning on Human Mental State [2.8246074016493457]
  We consider the human-aware task planning problem where a human-robot team is given a shared task with a known objective to achieve. Recent approaches tackle it by modeling it as a team of independent, rational agents, where the robot plans for both agents' (shared) tasks. We describe a novel approach to solve such problems, which models and uses execution-time observability conventions.
  arXiv  Detail & Related papers  (2022-10-17T09:21:00Z)
• Dynamic Human-Robot Role Allocation based on Human Ergonomics Risk Prediction and Robot Actions Adaptation [35.91053423341299]
  We propose a novel method that optimize assembly strategies and distribute the effort among the workers in human-robot cooperative tasks. The proposed approach succeeds in controlling the task allocation process to ensure safe and ergonomic conditions for the human worker.
  arXiv  Detail & Related papers  (2021-11-05T17:29:41Z)
• PHASE: PHysically-grounded Abstract Social Events for Machine Social Perception [50.551003004553806]
  We create a dataset of physically-grounded abstract social events, PHASE, that resemble a wide range of real-life social interactions. Phase is validated with human experiments demonstrating that humans perceive rich interactions in the social events. As a baseline model, we introduce a Bayesian inverse planning approach, SIMPLE, which outperforms state-of-the-art feed-forward neural networks.
  arXiv  Detail & Related papers  (2021-03-02T18:44:57Z)
• Watch-And-Help: A Challenge for Social Perception and Human-AI Collaboration [116.28433607265573]
  We introduce Watch-And-Help (WAH), a challenge for testing social intelligence in AI agents. In WAH, an AI agent needs to help a human-like agent perform a complex household task efficiently. We build VirtualHome-Social, a multi-agent household environment, and provide a benchmark including both planning and learning based baselines.
  arXiv  Detail & Related papers  (2020-10-19T21:48:31Z)
• AvE: Assistance via Empowerment [77.08882807208461]
  We propose a new paradigm for assistance by instead increasing the human's ability to control their environment. This task-agnostic objective preserves the person's autonomy and ability to achieve any eventual state.
  arXiv  Detail & Related papers  (2020-06-26T04:40:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.