Improved cooperation by balancing exploration and exploitation in intertemporal social dilemma tasks

• URL: http://arxiv.org/abs/2111.09152v1
• Date: Tue, 19 Oct 2021 08:40:56 GMT
• Title: Improved cooperation by balancing exploration and exploitation in intertemporal social dilemma tasks
• Authors: Zhenbo Cheng, Xingguang Liu, Leilei Zhang, Hangcheng Meng, Qin Li, Xiao Gang
• Abstract summary: We propose a new learning strategy for achieving coordination by incorporating a learning rate that can balance exploration and exploitation. We show that agents that use the simple strategy improve a relatively collective return in a decision task called the intertemporal social dilemma. We also explore the effects of the diversity of learning rates on the population of reinforcement learning agents and show that agents trained in heterogeneous populations develop particularly coordinated policies.
• Score: 2.541277269153809
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: When an individual's behavior has rational characteristics, this may lead to irrational collective actions for the group. A wide range of organisms from animals to humans often evolve the social attribute of cooperation to meet this challenge. Therefore, cooperation among individuals is of great significance for allowing social organisms to adapt to changes in the natural environment. Based on multi-agent reinforcement learning, we propose a new learning strategy for achieving coordination by incorporating a learning rate that can balance exploration and exploitation. We demonstrate that agents that use the simple strategy improve a relatively collective return in a decision task called the intertemporal social dilemma, where the conflict between the individual and the group is particularly sharp. We also explore the effects of the diversity of learning rates on the population of reinforcement learning agents and show that agents trained in heterogeneous populations develop particularly coordinated policies relative to those trained in homogeneous populations.

Related papers

• Multi-agent cooperation through learning-aware policy gradients [53.63948041506278]
  Self-interested individuals often fail to cooperate, posing a fundamental challenge for multi-agent learning. We present the first unbiased, higher-derivative-free policy gradient algorithm for learning-aware reinforcement learning. We derive from the iterated prisoner's dilemma a novel explanation for how and when cooperation arises among self-interested learning-aware agents.
  arXiv  Detail & Related papers  (2024-10-24T10:48:42Z)
• Learning to Balance Altruism and Self-interest Based on Empathy in Mixed-Motive Games [47.8980880888222]
  Multi-agent scenarios often involve mixed motives, demanding altruistic agents capable of self-protection against potential exploitation. We propose LASE Learning to balance Altruism and Self-interest based on Empathy. LASE allocates a portion of its rewards to co-players as gifts, with this allocation adapting dynamically based on the social relationship.
  arXiv  Detail & Related papers  (2024-10-10T12:30:56Z)
• Enhancing Cooperation through Selective Interaction and Long-term Experiences in Multi-Agent Reinforcement Learning [10.932974027102619]
  This study introduces a computational framework based on multi-agent reinforcement learning in the spatial Prisoner's Dilemma game. By modelling each agent using two distinct Q-networks, we disentangle the coevolutionary dynamics between cooperation and interaction.
  arXiv  Detail & Related papers  (2024-05-04T12:42:55Z)
• Dynamics of Moral Behavior in Heterogeneous Populations of Learning Agents [3.7414804164475983]
  We study the learning dynamics of morally heterogeneous populations interacting in a social dilemma setting. We observe several types of non-trivial interactions between pro-social and anti-social agents. We find that certain types of moral agents are able to steer selfish agents towards more cooperative behavior.
  arXiv  Detail & Related papers  (2024-03-07T04:12:24Z)
• Adaptive Coordination in Social Embodied Rearrangement [49.35582108902819]
  We study zero-shot coordination (ZSC) in this task, where an agent collaborates with a new partner, emulating a scenario where a robot collaborates with a new human partner. We propose Behavior Diversity Play (BDP), a novel ZSC approach that encourages diversity through a discriminability objective. Our results demonstrate that BDP learns adaptive agents that can tackle visual coordination, and zero-shot generalize to new partners in unseen environments, achieving 35% higher success and 32% higher efficiency compared to baselines.
  arXiv  Detail & Related papers  (2023-05-31T18:05:51Z)
• Learning Roles with Emergent Social Value Orientations [49.16026283952117]
  This paper introduces the typical "division of labor or roles" mechanism in human society. We provide a promising solution for intertemporal social dilemmas (ISD) with social value orientations (SVO) A novel learning framework, called Learning Roles with Emergent SVOs (RESVO), is proposed to transform the learning of roles into the social value orientation emergence.
  arXiv  Detail & Related papers  (2023-01-31T17:54:09Z)
• Flexible social inference facilitates targeted social learning when rewards are not observable [58.762004496858836]
  Groups coordinate more effectively when individuals are able to learn from others' successes. We suggest that social inference capacities may help bridge this gap, allowing individuals to update their beliefs about others' underlying knowledge and success from observable trajectories of behavior.
  arXiv  Detail & Related papers  (2022-12-01T21:04:03Z)
• Intrinsic fluctuations of reinforcement learning promote cooperation [0.0]
  Cooperating in social dilemma situations is vital for animals, humans, and machines. We demonstrate which and how individual elements of the multi-agent learning setting lead to cooperation.
  arXiv  Detail & Related papers  (2022-09-01T09:14:47Z)
• The art of compensation: how hybrid teams solve collective risk dilemmas [6.081979963786028]
  We study the evolutionary dynamics of cooperation in a hybrid population made of both adaptive and fixed-behavior agents. We show how the first learn to adapt their behavior to compensate for the behavior of the latter.
  arXiv  Detail & Related papers  (2022-05-13T13:23:42Z)
• Learning Collective Action under Risk Diversity [68.88688248278102]
  We investigate the consequences of risk diversity in groups of agents learning to play collective risk dilemmas. We show that risk diversity significantly reduces overall cooperation and hinders collective target achievement. Our results highlight the need for aligning risk perceptions among agents or develop new learning techniques.
  arXiv  Detail & Related papers  (2022-01-30T18:21:21Z)
• Deep reinforcement learning models the emergent dynamics of human cooperation [13.425401489679583]
  Experimental research has been unable to shed light on how social cognitive mechanisms contribute to the where and when of collective action. We leverage multi-agent deep reinforcement learning to model how a social-cognitive mechanism--specifically, the intrinsic motivation to achieve a good reputation--steers group behavior.
  arXiv  Detail & Related papers  (2021-03-08T18:58:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.