Open Problems in Cooperative AI

• URL: http://arxiv.org/abs/2012.08630v1
• Date: Tue, 15 Dec 2020 21:39:50 GMT
• Title: Open Problems in Cooperative AI
• Authors: Allan Dafoe and Edward Hughes and Yoram Bachrach and Tantum Collins and Kevin R. McKee and Joel Z. Leibo and Kate Larson and Thore Graepel
• Abstract summary: Research aims to study the many aspects of the problems of cooperation and to innovate in AI to contribute to solving these problems. This research integrates ongoing work on multi-agent systems, game theory and social choice, human-machine interaction and alignment, natural-language processing, and the construction of social tools and platforms.
• Score: 21.303564222227727
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Problems of cooperation--in which agents seek ways to jointly improve their welfare--are ubiquitous and important. They can be found at scales ranging from our daily routines--such as driving on highways, scheduling meetings, and working collaboratively--to our global challenges--such as peace, commerce, and pandemic preparedness. Arguably, the success of the human species is rooted in our ability to cooperate. Since machines powered by artificial intelligence are playing an ever greater role in our lives, it will be important to equip them with the capabilities necessary to cooperate and to foster cooperation. We see an opportunity for the field of artificial intelligence to explicitly focus effort on this class of problems, which we term Cooperative AI. The objective of this research would be to study the many aspects of the problems of cooperation and to innovate in AI to contribute to solving these problems. Central goals include building machine agents with the capabilities needed for cooperation, building tools to foster cooperation in populations of (machine and/or human) agents, and otherwise conducting AI research for insight relevant to problems of cooperation. This research integrates ongoing work on multi-agent systems, game theory and social choice, human-machine interaction and alignment, natural-language processing, and the construction of social tools and platforms. However, Cooperative AI is not the union of these existing areas, but rather an independent bet about the productivity of specific kinds of conversations that involve these and other areas. We see opportunity to more explicitly focus on the problem of cooperation, to construct unified theory and vocabulary, and to build bridges with adjacent communities working on cooperation, including in the natural, social, and behavioural sciences.

Related papers

• Constrained Human-AI Cooperation: An Inclusive Embodied Social Intelligence Challenge [47.74313897705183]
  CHAIC is an inclusive embodied social intelligence challenge designed to test social perception and cooperation in embodied agents. In CHAIC, the goal is for an embodied agent equipped with egocentric observations to assist a human who may be operating under physical constraints. We benchmark planning- and learning-based baselines on the challenge and introduce a new method that leverages large language models and behavior modeling.
  arXiv  Detail & Related papers  (2024-11-04T04:41:12Z)
• Mutual Theory of Mind in Human-AI Collaboration: An Empirical Study with LLM-driven AI Agents in a Real-time Shared Workspace Task [56.92961847155029]
  Theory of Mind (ToM) significantly impacts human collaboration and communication as a crucial capability to understand others. Mutual Theory of Mind (MToM) arises when AI agents with ToM capability collaborate with humans. We find that the agent's ToM capability does not significantly impact team performance but enhances human understanding of the agent.
  arXiv  Detail & Related papers  (2024-09-13T13:19:48Z)
• Problem Solving Through Human-AI Preference-Based Cooperation [74.39233146428492]
  We propose HAI-Co2, a novel human-AI co-construction framework. We formalize HAI-Co2 and discuss the difficult open research problems that it faces. We present a case study of HAI-Co2 and demonstrate its efficacy compared to monolithic generative AI models.
  arXiv  Detail & Related papers  (2024-08-14T11:06:57Z)
• Multi-Agent, Human-Agent and Beyond: A Survey on Cooperation in Social Dilemmas [15.785674974107204]
  The study of cooperation within social dilemmas has long been a fundamental topic across various disciplines. Recent advancements in Artificial Intelligence have significantly reshaped this field. This survey examines three key areas at the intersection of AI and cooperation in social dilemmas.
  arXiv  Detail & Related papers  (2024-02-27T07:31:30Z)
• Human AI Collaboration in Software Engineering: Lessons Learned from a Hands On Workshop [1.14603174659129]
  The study identifies key themes such as the evolving nature of human AI interaction, the capabilities of AI in software engineering tasks, and the challenges and limitations of integrating AI in this domain. The findings show that while AI, particularly ChatGPT, improves the efficiency of code generation and optimization, human oversight remains crucial, especially in areas requiring complex problem solving and security considerations.
  arXiv  Detail & Related papers  (2023-12-17T06:31:05Z)
• SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents [107.4138224020773]
  We present SOTOPIA, an open-ended environment to simulate complex social interactions between artificial agents and humans. In our environment, agents role-play and interact under a wide variety of scenarios; they coordinate, collaborate, exchange, and compete with each other to achieve complex social goals. We find that GPT-4 achieves a significantly lower goal completion rate than humans and struggles to exhibit social commonsense reasoning and strategic communication skills.
  arXiv  Detail & Related papers  (2023-10-18T02:27:01Z)
• CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society [58.04479313658851]
  This paper explores the potential of building scalable techniques to facilitate autonomous cooperation among communicative agents. We propose a novel communicative agent framework named role-playing. Our contributions include introducing a novel communicative agent framework, offering a scalable approach for studying the cooperative behaviors and capabilities of multi-agent systems.
  arXiv  Detail & Related papers  (2023-03-31T01:09:00Z)
• The art of compensation: how hybrid teams solve collective risk dilemmas [6.081979963786028]
  We study the evolutionary dynamics of cooperation in a hybrid population made of both adaptive and fixed-behavior agents. We show how the first learn to adapt their behavior to compensate for the behavior of the latter.
  arXiv  Detail & Related papers  (2022-05-13T13:23:42Z)
• Adversarial Attacks in Cooperative AI [0.0]
  Single-agent reinforcement learning algorithms in a multi-agent environment are inadequate for fostering cooperation. Recent work in adversarial machine learning shows that models can be easily deceived into making incorrect decisions. Cooperative AI might introduce new weaknesses not investigated in previous machine learning research.
  arXiv  Detail & Related papers  (2021-11-29T07:34:12Z)
• Artificial Intelligence & Cooperation [38.19500588776648]
  The rise of Artificial Intelligence will bring with it an ever-increasing willingness to cede decision-making to machines. But rather than just giving machines the power to make decisions that affect us, we need ways to work cooperatively with AI systems. With success, cooperation between humans and AIs can build society just as human-human cooperation has.
  arXiv  Detail & Related papers  (2020-12-10T23:54:31Z)
• Watch-And-Help: A Challenge for Social Perception and Human-AI Collaboration [116.28433607265573]
  We introduce Watch-And-Help (WAH), a challenge for testing social intelligence in AI agents. In WAH, an AI agent needs to help a human-like agent perform a complex household task efficiently. We build VirtualHome-Social, a multi-agent household environment, and provide a benchmark including both planning and learning based baselines.
  arXiv  Detail & Related papers  (2020-10-19T21:48:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.