Learning Collective Action under Risk Diversity

• URL: http://arxiv.org/abs/2201.12891v1
• Date: Sun, 30 Jan 2022 18:21:21 GMT
• Title: Learning Collective Action under Risk Diversity
• Authors: Ramona Merhej, Fernando P. Santos, Francisco S. Melo, Mohamed Chetouani, Francisco C. Santos
• Abstract summary: We investigate the consequences of risk diversity in groups of agents learning to play collective risk dilemmas. We show that risk diversity significantly reduces overall cooperation and hinders collective target achievement. Our results highlight the need for aligning risk perceptions among agents or develop new learning techniques.
• Score: 68.88688248278102
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Collective risk dilemmas (CRDs) are a class of n-player games that represent societal challenges where groups need to coordinate to avoid the risk of a disastrous outcome. Multi-agent systems incurring such dilemmas face difficulties achieving cooperation and often converge to sub-optimal, risk-dominant solutions where everyone defects. In this paper we investigate the consequences of risk diversity in groups of agents learning to play CRDs. We find that risk diversity places new challenges to cooperation that are not observed in homogeneous groups. We show that increasing risk diversity significantly reduces overall cooperation and hinders collective target achievement. It leads to asymmetrical changes in agents' policies -- i.e. the increase in contributions from individuals at high risk is unable to compensate for the decrease in contributions from individuals at low risk -- which overall reduces the total contributions in a population. When comparing RL behaviors to rational individualistic and social behaviors, we find that RL populations converge to fairer contributions among agents. Our results highlight the need for aligning risk perceptions among agents or develop new learning techniques that explicitly account for risk diversity.

Related papers

• Risk-Sensitive Multi-Agent Reinforcement Learning in Network Aggregative Markov Games [2.85386288555414]
  We propose a distributed sampling-based actor-critic (AC) algorithm with CPT risk for network aggregative games (NAMGs) Under a set of assumptions, we prove to a subjective notion of perfect Nash equilibrium in NAMGs. Experiments show that subjective policies can be different from the risk-neutral ones.
  arXiv  Detail & Related papers  (2024-02-08T18:43:27Z)
• RiskQ: Risk-sensitive Multi-Agent Reinforcement Learning Value Factorization [49.26510528455664]
  We introduce the Risk-sensitive Individual-Global-Max (RIGM) principle as a generalization of the Individual-Global-Max (IGM) and Distributional IGM (DIGM) principles. We show that RiskQ can obtain promising performance through extensive experiments.
  arXiv  Detail & Related papers  (2023-11-03T07:18:36Z)
• Provably Efficient Iterated CVaR Reinforcement Learning with Function Approximation and Human Feedback [57.6775169085215]
  Risk-sensitive reinforcement learning aims to optimize policies that balance the expected reward and risk. We present a novel framework that employs an Iterated Conditional Value-at-Risk (CVaR) objective under both linear and general function approximations. We propose provably sample-efficient algorithms for this Iterated CVaR RL and provide rigorous theoretical analysis.
  arXiv  Detail & Related papers  (2023-07-06T08:14:54Z)
• Anticipatory Thinking Challenges in Open Worlds: Risk Management [7.820667552233988]
  As AI systems become part of everyday life, they too have begun to manage risk. Learning to identify and mitigate low-frequency, high-impact risks is at odds with the observational bias required to train machine learning models. Our goal is to spur research into solutions that assess and improve the anticipatory thinking required by AI agents to manage risk in open-worlds and ultimately the real-world.
  arXiv  Detail & Related papers  (2023-06-22T18:31:17Z)
• Social Diversity Reduces the Complexity and Cost of Fostering Fairness [63.70639083665108]
  We investigate the effects of interference mechanisms which assume incomplete information and flexible standards of fairness. We quantify the role of diversity and show how it reduces the need for information gathering. Our results indicate that diversity changes and opens up novel mechanisms available to institutions wishing to promote fairness.
  arXiv  Detail & Related papers  (2022-11-18T21:58:35Z)
• Learning Risk-Averse Equilibria in Multi-Agent Systems [13.25454171233235]
  In multi-agent systems, intelligent agents are tasked with making decisions that have optimal outcomes when the actions of the other agents are as expected. We introduce a new risk-averse solution concept that allows the learner to accommodate unexpected actions. We show that our population of agents that approximate a risk-averse equilibrium is particularly effective in the presence of unseen opposing populations.
  arXiv  Detail & Related papers  (2022-05-30T21:20:30Z)
• A Survey of Risk-Aware Multi-Armed Bandits [84.67376599822569]
  We review various risk measures of interest, and comment on their properties. We consider algorithms for the regret minimization setting, where the exploration-exploitation trade-off manifests. We conclude by commenting on persisting challenges and fertile areas for future research.
  arXiv  Detail & Related papers  (2022-05-12T02:20:34Z)
• Efficient Risk-Averse Reinforcement Learning [79.61412643761034]
  In risk-averse reinforcement learning (RL), the goal is to optimize some risk measure of the returns. We prove that under certain conditions this inevitably leads to a local-optimum barrier, and propose a soft risk mechanism to bypass it. We demonstrate improved risk aversion in maze navigation, autonomous driving, and resource allocation benchmarks.
  arXiv  Detail & Related papers  (2022-05-10T19:40:52Z)
• Improved cooperation by balancing exploration and exploitation in intertemporal social dilemma tasks [2.541277269153809]
  We propose a new learning strategy for achieving coordination by incorporating a learning rate that can balance exploration and exploitation. We show that agents that use the simple strategy improve a relatively collective return in a decision task called the intertemporal social dilemma. We also explore the effects of the diversity of learning rates on the population of reinforcement learning agents and show that agents trained in heterogeneous populations develop particularly coordinated policies.
  arXiv  Detail & Related papers  (2021-10-19T08:40:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.