Understanding the World to Solve Social Dilemmas Using Multi-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2305.11358v1
• Date: Fri, 19 May 2023 00:31:26 GMT
• Title: Understanding the World to Solve Social Dilemmas Using Multi-Agent Reinforcement Learning
• Authors: Manuel Rios, Nicanor Quijano, Luis Felipe Giraldo
• Abstract summary: We study the behavior of self-interested rational agents that learn world models in a multi-agent reinforcement learning setting. Our simulation results show that groups of agents endowed with world models outperform all the other tested ones when dealing with scenarios where social dilemmas can arise.
• Score: 0.7161783472741748
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Social dilemmas are situations where groups of individuals can benefit from mutual cooperation but conflicting interests impede them from doing so. This type of situations resembles many of humanity's most critical challenges, and discovering mechanisms that facilitate the emergence of cooperative behaviors is still an open problem. In this paper, we study the behavior of self-interested rational agents that learn world models in a multi-agent reinforcement learning (RL) setting and that coexist in environments where social dilemmas can arise. Our simulation results show that groups of agents endowed with world models outperform all the other tested ones when dealing with scenarios where social dilemmas can arise. We exploit the world model architecture to qualitatively assess the learnt dynamics and confirm that each agent's world model is capable to encode information of the behavior of the changing environment and the other agent's actions. This is the first work that shows that world models facilitate the emergence of complex coordinated behaviors that enable interacting agents to ``understand'' both environmental and social dynamics.

Related papers

• Multi-agent cooperation through learning-aware policy gradients [53.63948041506278]
  Self-interested individuals often fail to cooperate, posing a fundamental challenge for multi-agent learning. We present the first unbiased, higher-derivative-free policy gradient algorithm for learning-aware reinforcement learning. We derive from the iterated prisoner's dilemma a novel explanation for how and when cooperation arises among self-interested learning-aware agents.
  arXiv  Detail & Related papers  (2024-10-24T10:48:42Z)
• The Dynamics of Social Conventions in LLM populations: Spontaneous Emergence, Collective Biases and Tipping Points [0.0]
  We investigate the dynamics of conventions within populations of Large Language Model (LLM) agents using simulated interactions. We show that globally accepted social conventions can spontaneously arise from local interactions between communicating LLMs. Minority groups of committed LLMs can drive social change by establishing new social conventions.
  arXiv  Detail & Related papers  (2024-10-11T16:16:38Z)
• SocialGFs: Learning Social Gradient Fields for Multi-Agent Reinforcement Learning [58.84311336011451]
  We propose a novel gradient-based state representation for multi-agent reinforcement learning. We employ denoising score matching to learn the social gradient fields (SocialGFs) from offline samples. In practice, we integrate SocialGFs into the widely used multi-agent reinforcement learning algorithms, e.g., MAPPO.
  arXiv  Detail & Related papers  (2024-05-03T04:12:19Z)
• COMBO: Compositional World Models for Embodied Multi-Agent Cooperation [64.27636858152522]
  Decentralized agents must cooperate given only partial egocentric views of the world. We train generative models to estimate the overall world state given partial egocentric observations. We learn a compositional world model for multi-agent cooperation by factorizing the naturally composable joint actions of multiple agents.
  arXiv  Detail & Related papers  (2024-04-16T17:59:11Z)
• Dynamics of Moral Behavior in Heterogeneous Populations of Learning Agents [3.7414804164475983]
  We study the learning dynamics of morally heterogeneous populations interacting in a social dilemma setting. We observe several types of non-trivial interactions between pro-social and anti-social agents. We find that certain types of moral agents are able to steer selfish agents towards more cooperative behavior.
  arXiv  Detail & Related papers  (2024-03-07T04:12:24Z)
• SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents [107.4138224020773]
  We present SOTOPIA, an open-ended environment to simulate complex social interactions between artificial agents and humans. In our environment, agents role-play and interact under a wide variety of scenarios; they coordinate, collaborate, exchange, and compete with each other to achieve complex social goals. We find that GPT-4 achieves a significantly lower goal completion rate than humans and struggles to exhibit social commonsense reasoning and strategic communication skills.
  arXiv  Detail & Related papers  (2023-10-18T02:27:01Z)
• Flexible social inference facilitates targeted social learning when rewards are not observable [58.762004496858836]
  Groups coordinate more effectively when individuals are able to learn from others' successes. We suggest that social inference capacities may help bridge this gap, allowing individuals to update their beliefs about others' underlying knowledge and success from observable trajectories of behavior.
  arXiv  Detail & Related papers  (2022-12-01T21:04:03Z)
• Modelling Cooperation in Network Games with Spatio-Temporal Complexity [11.665246332943058]
  We study the emergence of self-organized cooperation in complex gridworld domains. Using multi-agent deep reinforcement learning, we simulate an agent society for a variety of plausible mechanisms. Our methods have implications for mechanism design in both human and artificial agent systems.
  arXiv  Detail & Related papers  (2021-02-13T12:04:52Z)
• Emergent Reciprocity and Team Formation from Randomized Uncertain Social Preferences [8.10414043447031]
  We show evidence of emergent direct reciprocity, indirect reciprocity and reputation, and team formation when training agents with randomized uncertain social preferences (RUSP) RUSP is generic and scalable; it can be applied to any multi-agent environment without changing the original underlying game dynamics or objectives. In particular, we show that with RUSP these behaviors can emerge and lead to higher social welfare equilibria in both classic abstract social dilemmas like Iterated Prisoner's Dilemma as well in more complex intertemporal environments.
  arXiv  Detail & Related papers  (2020-11-10T20:06:19Z)
• Emergent Social Learning via Multi-agent Reinforcement Learning [91.57176641192771]
  Social learning is a key component of human and animal intelligence. This paper investigates whether independent reinforcement learning agents can learn to use social learning to improve their performance.
  arXiv  Detail & Related papers  (2020-10-01T17:54:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.