The State-Action-Reward-State-Action Algorithm in Spatial Prisoner's Dilemma Game
- URL: http://arxiv.org/abs/2406.17326v1
- Date: Tue, 25 Jun 2024 07:21:35 GMT
- Title: The State-Action-Reward-State-Action Algorithm in Spatial Prisoner's Dilemma Game
- Authors: Lanyu Yang, Dongchun Jiang, Fuqiang Guo, Mingjian Fu,
- Abstract summary: Reinforcement learning provides a suitable framework for studying evolutionary game theory.
We employ the State-Action-Reward-State-Action algorithm as the decision-making mechanism for individuals in evolutionary game theory.
We evaluate the impact of SARSA on cooperation rates by analyzing variations in rewards and the distribution of cooperators and defectors within the network.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Cooperative behavior is prevalent in both human society and nature. Understanding the emergence and maintenance of cooperation among self-interested individuals remains a significant challenge in evolutionary biology and social sciences. Reinforcement learning (RL) provides a suitable framework for studying evolutionary game theory as it can adapt to environmental changes and maximize expected benefits. In this study, we employ the State-Action-Reward-State-Action (SARSA) algorithm as the decision-making mechanism for individuals in evolutionary game theory. Initially, we apply SARSA to imitation learning, where agents select neighbors to imitate based on rewards. This approach allows us to observe behavioral changes in agents without independent decision-making abilities. Subsequently, SARSA is utilized for primary agents to independently choose cooperation or betrayal with their neighbors. We evaluate the impact of SARSA on cooperation rates by analyzing variations in rewards and the distribution of cooperators and defectors within the network.
Related papers
- Bias Mitigation via Compensation: A Reinforcement Learning Perspective [1.5442389863546546]
Group dynamics might require that one agent (e.g., the AI system) compensate for biases and errors in another agent (e.g., the human)
We provide a theoretical framework for algorithmic compensation that synthesizes game theory and reinforcement learning principles.
This work then underpins our ethical analysis of the conditions in which AI agents should adapt to biases and behaviors of other agents.
arXiv Detail & Related papers (2024-04-30T04:41:47Z) - Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents [101.17919953243107]
GovSim is a generative simulation platform designed to study strategic interactions and cooperative decision-making in large language models (LLMs)
We find that all but the most powerful LLM agents fail to achieve a sustainable equilibrium in GovSim, with the highest survival rate below 54%.
We show that agents that leverage "Universalization"-based reasoning, a theory of moral thinking, are able to achieve significantly better sustainability.
arXiv Detail & Related papers (2024-04-25T15:59:16Z) - Learning Roles with Emergent Social Value Orientations [49.16026283952117]
This paper introduces the typical "division of labor or roles" mechanism in human society.
We provide a promising solution for intertemporal social dilemmas (ISD) with social value orientations (SVO)
A novel learning framework, called Learning Roles with Emergent SVOs (RESVO), is proposed to transform the learning of roles into the social value orientation emergence.
arXiv Detail & Related papers (2023-01-31T17:54:09Z) - On Blockchain We Cooperate: An Evolutionary Game Perspective [0.8566457170664925]
In this paper, we introduce rationality and game-theoretical solution concepts to study the equilibrium outcomes of consensus protocols.
We apply bounded rationality to model agent behavior, and resolve the initial conditions for three different stable equilibria.
Our research contributes to the literature across disciplines, including distributed consensus in computer science, game theory in economics on blockchain consensus, evolutionary game theory at the intersection of biology and economics, and cooperative AI with joint insights into computing and social science.
arXiv Detail & Related papers (2022-12-10T19:56:10Z) - Incorporating Rivalry in Reinforcement Learning for a Competitive Game [65.2200847818153]
This work proposes a novel reinforcement learning mechanism based on the social impact of rivalry behavior.
Our proposed model aggregates objective and social perception mechanisms to derive a rivalry score that is used to modulate the learning of artificial agents.
arXiv Detail & Related papers (2022-08-22T14:06:06Z) - Improved cooperation by balancing exploration and exploitation in
intertemporal social dilemma tasks [2.541277269153809]
We propose a new learning strategy for achieving coordination by incorporating a learning rate that can balance exploration and exploitation.
We show that agents that use the simple strategy improve a relatively collective return in a decision task called the intertemporal social dilemma.
We also explore the effects of the diversity of learning rates on the population of reinforcement learning agents and show that agents trained in heterogeneous populations develop particularly coordinated policies.
arXiv Detail & Related papers (2021-10-19T08:40:56Z) - Birds of a Feather Flock Together: A Close Look at Cooperation Emergence
via Multi-Agent RL [20.22747008079794]
We study the dynamics of a second-order social dilemma resulting from incentivizing mechanisms.
We find that a typical tendency of humans, called homophily, can solve the problem.
We propose a novel learning framework to encourage incentive homophily.
arXiv Detail & Related papers (2021-04-23T08:00:45Z) - End-to-End Learning and Intervention in Games [60.41921763076017]
We provide a unified framework for learning and intervention in games.
We propose two approaches, respectively based on explicit and implicit differentiation.
The analytical results are validated using several real-world problems.
arXiv Detail & Related papers (2020-10-26T18:39:32Z) - Decentralized Reinforcement Learning: Global Decision-Making via Local
Economic Transactions [80.49176924360499]
We establish a framework for directing a society of simple, specialized, self-interested agents to solve sequential decision problems.
We derive a class of decentralized reinforcement learning algorithms.
We demonstrate the potential advantages of a society's inherent modular structure for more efficient transfer learning.
arXiv Detail & Related papers (2020-07-05T16:41:09Z) - Cooperative Inverse Reinforcement Learning [64.60722062217417]
We propose a formal definition of the value alignment problem as cooperative reinforcement learning (CIRL)
A CIRL problem is a cooperative, partial-information game with two agents human and robot; both are rewarded according to the human's reward function, but the robot does not initially know what this is.
In contrast to classical IRL, where the human is assumed to act optimally in isolation, optimal CIRL solutions produce behaviors such as active teaching, active learning, and communicative actions.
arXiv Detail & Related papers (2016-06-09T22:39:54Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.