Birds of a Feather Flock Together: A Close Look at Cooperation Emergence via Multi-Agent RL

• URL: http://arxiv.org/abs/2104.11455v1
• Date: Fri, 23 Apr 2021 08:00:45 GMT
• Title: Birds of a Feather Flock Together: A Close Look at Cooperation Emergence via Multi-Agent RL
• Authors: Heng Dong, Tonghan Wang, Jiayuan Liu, Chongjie Zhang
• Abstract summary: We study the dynamics of a second-order social dilemma resulting from incentivizing mechanisms. We find that a typical tendency of humans, called homophily, can solve the problem. We propose a novel learning framework to encourage incentive homophily.
• Score: 20.22747008079794
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: How cooperation emerges is a long-standing and interdisciplinary problem. Game-theoretical studies on social dilemmas reveal that altruistic incentives are critical to the emergence of cooperation but their analyses are limited to stateless games. For more realistic scenarios, multi-agent reinforcement learning has been used to study sequential social dilemmas (SSDs). Recent works show that learning to incentivize other agents can promote cooperation in SSDs. However, with these incentivizing mechanisms, the team cooperation level does not converge and regularly oscillates between cooperation and defection during learning. We show that a second-order social dilemma resulting from these incentive mechanisms is the main reason for such fragile cooperation. We analyze the dynamics of this second-order social dilemma and find that a typical tendency of humans, called homophily, can solve the problem. We propose a novel learning framework to encourage incentive homophily and show that it achieves stable cooperation in both public goods dilemma and tragedy of the commons dilemma.

Related papers

• Multi-agent cooperation through learning-aware policy gradients [53.63948041506278]
  Self-interested individuals often fail to cooperate, posing a fundamental challenge for multi-agent learning. We present the first unbiased, higher-derivative-free policy gradient algorithm for learning-aware reinforcement learning. We derive from the iterated prisoner's dilemma a novel explanation for how and when cooperation arises among self-interested learning-aware agents.
  arXiv  Detail & Related papers  (2024-10-24T10:48:42Z)
• Learning to Balance Altruism and Self-interest Based on Empathy in Mixed-Motive Games [47.8980880888222]
  Multi-agent scenarios often involve mixed motives, demanding altruistic agents capable of self-protection against potential exploitation. We propose LASE Learning to balance Altruism and Self-interest based on Empathy. LASE allocates a portion of its rewards to co-players as gifts, with this allocation adapting dynamically based on the social relationship.
  arXiv  Detail & Related papers  (2024-10-10T12:30:56Z)
• Multi-Agent, Human-Agent and Beyond: A Survey on Cooperation in Social Dilemmas [15.785674974107204]
  The study of cooperation within social dilemmas has long been a fundamental topic across various disciplines. Recent advancements in Artificial Intelligence have significantly reshaped this field. This survey examines three key areas at the intersection of AI and cooperation in social dilemmas.
  arXiv  Detail & Related papers  (2024-02-27T07:31:30Z)
• Deconstructing Cooperation and Ostracism via Multi-Agent Reinforcement Learning [3.3751859064985483]
  We show that network rewiring facilitates mutual cooperation even when one agent always offers cooperation. We also find that ostracism alone is not sufficient to make cooperation emerge. Our findings provide insights into the conditions and mechanisms necessary for the emergence of cooperation.
  arXiv  Detail & Related papers  (2023-10-06T23:18:55Z)
• Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View [60.80731090755224]
  This paper probes the collaboration mechanisms among contemporary NLP systems by practical experiments with theoretical insights. We fabricate four unique societies' comprised of LLM agents, where each agent is characterized by a specific trait' (easy-going or overconfident) and engages in collaboration with a distinct thinking pattern' (debate or reflection) Our results further illustrate that LLM agents manifest human-like social behaviors, such as conformity and consensus reaching, mirroring social psychology theories.
  arXiv  Detail & Related papers  (2023-10-03T15:05:52Z)
• Learning Roles with Emergent Social Value Orientations [49.16026283952117]
  This paper introduces the typical "division of labor or roles" mechanism in human society. We provide a promising solution for intertemporal social dilemmas (ISD) with social value orientations (SVO) A novel learning framework, called Learning Roles with Emergent SVOs (RESVO), is proposed to transform the learning of roles into the social value orientation emergence.
  arXiv  Detail & Related papers  (2023-01-31T17:54:09Z)
• Investigating the Impact of Direct Punishment on the Emergence of Cooperation in Multi-Agent Reinforcement Learning Systems [2.4555276449137042]
  Problems of cooperation are omnipresent within human society. As the use of AI becomes more pervasive throughout society, the need for socially intelligent agents is becoming increasingly evident. This paper presents a comprehensive analysis and evaluation of the behaviors and learning dynamics associated with direct punishment, third-party punishment, partner selection, and reputation.
  arXiv  Detail & Related papers  (2023-01-19T19:33:54Z)
• Intrinsic fluctuations of reinforcement learning promote cooperation [0.0]
  Cooperating in social dilemma situations is vital for animals, humans, and machines. We demonstrate which and how individual elements of the multi-agent learning setting lead to cooperation.
  arXiv  Detail & Related papers  (2022-09-01T09:14:47Z)
• Tackling Asymmetric and Circular Sequential Social Dilemmas with Reinforcement Learning and Graph-based Tit-for-Tat [0.0]
  Social dilemmas offer situations where multiple actors should all cooperate to achieve the best outcome but greed and fear lead to a worst self-interested issue. Recently, the emergence of Deep Reinforcement Learning has generated revived interest in social dilemmas with the introduction of Sequential Social Dilemma (SSD) This paper extends SSD with Circular Sequential Social Dilemma (CSSD), a new kind of Markov games that better generalizes the diversity of cooperation between agents.
  arXiv  Detail & Related papers  (2022-06-26T15:42:48Z)
• Hidden Agenda: a Social Deduction Game with Diverse Learned Equilibria [57.74495091445414]
  Social deduction games offer an avenue to study how individuals might learn to synthesize potentially unreliable information about others. In this work, we present Hidden Agenda, a two-team social deduction game that provides a 2D environment for studying learning agents in scenarios of unknown team alignment. Reinforcement learning agents trained in Hidden Agenda show that agents can learn a variety of behaviors, including partnering and voting without need for communication in natural language.
  arXiv  Detail & Related papers  (2022-01-05T20:54:10Z)
• Incorporating Rivalry in Reinforcement Learning for a Competitive Game [65.2200847818153]
  This study focuses on providing a novel learning mechanism based on a rivalry social impact. Based on the concept of competitive rivalry, our analysis aims to investigate if we can change the assessment of these agents from a human perspective.
  arXiv  Detail & Related papers  (2020-11-02T21:54:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.