Using deep reinforcement learning to promote sustainable human behaviour on a common pool resource problem

• URL: http://arxiv.org/abs/2404.15059v1
• Date: Tue, 23 Apr 2024 14:07:39 GMT
• Title: Using deep reinforcement learning to promote sustainable human behaviour on a common pool resource problem
• Authors: Raphael Koster, Miruna Pîslar, Andrea Tacchetti, Jan Balaguer, Leqi Liu, Romuald Elie, Oliver P. Hauser, Karl Tuyls, Matt Botvinick, Christopher Summerfield,
• Abstract summary: We design an allocation mechanism that endogenously promotes sustainable contributions from human participants to a common pool resource. Deep reinforcement learning can be used to discover mechanisms that promote sustainable human behaviour.
• Score: 14.272912268098375
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A canonical social dilemma arises when finite resources are allocated to a group of people, who can choose to either reciprocate with interest, or keep the proceeds for themselves. What resource allocation mechanisms will encourage levels of reciprocation that sustain the commons? Here, in an iterated multiplayer trust game, we use deep reinforcement learning (RL) to design an allocation mechanism that endogenously promotes sustainable contributions from human participants to a common pool resource. We first trained neural networks to behave like human players, creating a stimulated economy that allowed us to study how different mechanisms influenced the dynamics of receipt and reciprocation. We then used RL to train a social planner to maximise aggregate return to players. The social planner discovered a redistributive policy that led to a large surplus and an inclusive economy, in which players made roughly equal gains. The RL agent increased human surplus over baseline mechanisms based on unrestricted welfare or conditional cooperation, by conditioning its generosity on available resources and temporarily sanctioning defectors by allocating fewer resources to them. Examining the AI policy allowed us to develop an explainable mechanism that performed similarly and was more popular among players. Deep reinforcement learning can be used to discover mechanisms that promote sustainable human behaviour.

Related papers

• Regularized Proportional Fairness Mechanism for Resource Allocation Without Money [18.593061465167363]
  We design an innovative neural network architecture tailored to the resource allocation problem, which we name Regularized Proportional Fairness Network (RPF-Net) RPF-Net regularizes the output of the proportional fairness (PF) mechanism by a learned function approximator of the most exploitable allocation. We derive generalization bounds that guarantee the mechanism performance when trained under finite and out-of-distribution samples.
  arXiv  Detail & Related papers  (2025-01-02T07:17:23Z)
• Learning to Balance Altruism and Self-interest Based on Empathy in Mixed-Motive Games [47.8980880888222]
  Multi-agent scenarios often involve mixed motives, demanding altruistic agents capable of self-protection against potential exploitation. We propose LASE Learning to balance Altruism and Self-interest based on Empathy. LASE allocates a portion of its rewards to co-players as gifts, with this allocation adapting dynamically based on the social relationship.
  arXiv  Detail & Related papers  (2024-10-10T12:30:56Z)
• The State-Action-Reward-State-Action Algorithm in Spatial Prisoner's Dilemma Game [0.0]
  Reinforcement learning provides a suitable framework for studying evolutionary game theory. We employ the State-Action-Reward-State-Action algorithm as the decision-making mechanism for individuals in evolutionary game theory. We evaluate the impact of SARSA on cooperation rates by analyzing variations in rewards and the distribution of cooperators and defectors within the network.
  arXiv  Detail & Related papers  (2024-06-25T07:21:35Z)
• Formal Contracts Mitigate Social Dilemmas in Multi-Agent RL [4.969697978555126]
  Multi-agent Reinforcement Learning (MARL) is a powerful tool for training autonomous agents acting independently in a common environment. MARL can lead to sub-optimal behavior when individual incentives and group incentives diverge. We propose an augmentation to a Markov game where agents voluntarily agree to binding transfers of reward, under pre-specified conditions.
  arXiv  Detail & Related papers  (2022-08-22T17:42:03Z)
• Learning Dynamic Mechanisms in Unknown Environments: A Reinforcement Learning Approach [123.55983746427572]
  We propose novel learning algorithms to recover the dynamic Vickrey-Clarke-Grove (VCG) mechanism over multiple rounds of interaction. A key contribution of our approach is incorporating reward-free online Reinforcement Learning (RL) to aid exploration over a rich policy space.
  arXiv  Detail & Related papers  (2022-02-25T16:17:23Z)
• Modeling Bounded Rationality in Multi-Agent Simulations Using Rationally Inattentive Reinforcement Learning [85.86440477005523]
  We study more human-like RL agents which incorporate an established model of human-irrationality, the Rational Inattention (RI) model. RIRL models the cost of cognitive information processing using mutual information. We show that using RIRL yields a rich spectrum of new equilibrium behaviors that differ from those found under rational assumptions.
  arXiv  Detail & Related papers  (2022-01-18T20:54:00Z)
• Explore and Control with Adversarial Surprise [78.41972292110967]
  Reinforcement learning (RL) provides a framework for learning goal-directed policies given user-specified rewards. We propose a new unsupervised RL technique based on an adversarial game which pits two policies against each other to compete over the amount of surprise an RL agent experiences. We show that our method leads to the emergence of complex skills by exhibiting clear phase transitions.
  arXiv  Detail & Related papers  (2021-07-12T17:58:40Z)
• Multi-Principal Assistance Games: Definition and Collegial Mechanisms [16.491889275389457]
  We introduce the concept of a multi-principal assistance game (MPAG) In an MPAG, a single agent assists N human principals who may have widely different preferences. We analyze in particular a generalization of apprenticeship learning in which the humans first perform some work to obtain utility and demonstrate their preferences.
  arXiv  Detail & Related papers  (2020-12-29T00:06:47Z)
• Coordinated Online Learning for Multi-Agent Systems with Coupled Constraints and Perturbed Utility Observations [91.02019381927236]
  We introduce a novel method to steer the agents toward a stable population state, fulfilling the given resource constraints. The proposed method is a decentralized resource pricing method based on the resource loads resulting from the augmentation of the game's Lagrangian.
  arXiv  Detail & Related papers  (2020-10-21T10:11:17Z)
• Emergent Social Learning via Multi-agent Reinforcement Learning [91.57176641192771]
  Social learning is a key component of human and animal intelligence. This paper investigates whether independent reinforcement learning agents can learn to use social learning to improve their performance.
  arXiv  Detail & Related papers  (2020-10-01T17:54:14Z)
• Multi-Principal Assistance Games [11.85513759444069]
  Impossibility theorems in social choice theory and voting theory can be applied to such games. We analyze in particular a bandit apprentice game in which the humans act first to demonstrate their individual preferences for the arms. We propose a social choice method that uses shared control of a system to combine preference inference with social welfare optimization.
  arXiv  Detail & Related papers  (2020-07-19T00:23:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.