Reward-Sharing Relational Networks in Multi-Agent Reinforcement Learning as a Framework for Emergent Behavior

• URL: http://arxiv.org/abs/2207.05886v2
• Date: Thu, 14 Jul 2022 01:02:46 GMT
• Title: Reward-Sharing Relational Networks in Multi-Agent Reinforcement Learning as a Framework for Emergent Behavior
• Authors: Hossein Haeri, Reza Ahmadzadeh, Kshitij Jerath
• Abstract summary: We integrate social' interactions into the MARL setup through a user-defined relational network. We examine the effects of agent-agent relations on the rise of emergent behaviors.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In this work, we integrate `social' interactions into the MARL setup through a user-defined relational network and examine the effects of agent-agent relations on the rise of emergent behaviors. Leveraging insights from sociology and neuroscience, our proposed framework models agent relationships using the notion of Reward-Sharing Relational Networks (RSRN), where network edge weights act as a measure of how much one agent is invested in the success of (or `cares about') another. We construct relational rewards as a function of the RSRN interaction weights to collectively train the multi-agent system via a multi-agent reinforcement learning algorithm. The performance of the system is tested for a 3-agent scenario with different relational network structures (e.g., self-interested, communitarian, and authoritarian networks). Our results indicate that reward-sharing relational networks can significantly influence learned behaviors. We posit that RSRN can act as a framework where different relational networks produce distinct emergent behaviors, often analogous to the intuited sociological understanding of such networks.

Related papers

• Adaptive Network Intervention for Complex Systems: A Hierarchical Graph Reinforcement Learning Approach [0.8287206589886879]
  This paper introduces a Hierarchical Graph Reinforcement Learning framework that governs such systems through targeted interventions in the network structure. Under low social learning, the HGRL manager preserves cooperation, forming robust core-periphery networks dominated by cooperators. In contrast, high social learning defection accelerates, leading to sparser, chain-like networks.
  arXiv  Detail & Related papers  (2024-10-30T18:59:02Z)
• Problem-Solving in Language Model Networks [44.99833362998488]
  This work extends the concept of multi-agent debate to more general network topologies. It measures the question-answering accuracy, influence, consensus, and the effects of bias on the collective.
  arXiv  Detail & Related papers  (2024-06-18T07:59:14Z)
• Scaling Large-Language-Model-based Multi-Agent Collaboration [75.5241464256688]
  Pioneering advancements in large language model-powered agents have underscored the design pattern of multi-agent collaboration. Inspired by the neural scaling law, this study investigates whether a similar principle applies to increasing agents in multi-agent collaboration.
  arXiv  Detail & Related papers  (2024-06-11T11:02:04Z)
• Enhancing Cooperation through Selective Interaction and Long-term Experiences in Multi-Agent Reinforcement Learning [10.932974027102619]
  This study introduces a computational framework based on multi-agent reinforcement learning in the spatial Prisoner's Dilemma game. By modelling each agent using two distinct Q-networks, we disentangle the coevolutionary dynamics between cooperation and interaction.
  arXiv  Detail & Related papers  (2024-05-04T12:42:55Z)
• SocialGFs: Learning Social Gradient Fields for Multi-Agent Reinforcement Learning [58.84311336011451]
  We propose a novel gradient-based state representation for multi-agent reinforcement learning. We employ denoising score matching to learn the social gradient fields (SocialGFs) from offline samples. In practice, we integrate SocialGFs into the widely used multi-agent reinforcement learning algorithms, e.g., MAPPO.
  arXiv  Detail & Related papers  (2024-05-03T04:12:19Z)
• Multi-Agent Dynamic Relational Reasoning for Social Robot Navigation [55.65482030032804]
  Social robot navigation can be helpful in various contexts of daily life but requires safe human-robot interactions and efficient trajectory planning. We propose a systematic relational reasoning approach with explicit inference of the underlying dynamically evolving relational structures. Our approach infers dynamically evolving relation graphs and hypergraphs to capture the evolution of relations, which the trajectory predictor employs to generate future states.
  arXiv  Detail & Related papers  (2024-01-22T18:58:22Z)
• Leveraging Low-Rank and Sparse Recurrent Connectivity for Robust Closed-Loop Control [63.310780486820796]
  We show how a parameterization of recurrent connectivity influences robustness in closed-loop settings. We find that closed-form continuous-time neural networks (CfCs) with fewer parameters can outperform their full-rank, fully-connected counterparts.
  arXiv  Detail & Related papers  (2023-10-05T21:44:18Z)
• Multi-Agent Imitation Learning with Copulas [102.27052968901894]
  Multi-agent imitation learning aims to train multiple agents to perform tasks from demonstrations by learning a mapping between observations and actions. In this paper, we propose to use copula, a powerful statistical tool for capturing dependence among random variables, to explicitly model the correlation and coordination in multi-agent systems. Our proposed model is able to separately learn marginals that capture the local behavioral patterns of each individual agent, as well as a copula function that solely and fully captures the dependence structure among agents.
  arXiv  Detail & Related papers  (2021-07-10T03:49:41Z)
• The Impact of Network Connectivity on Collective Learning [1.370633147306388]
  In decentralised autonomous systems it is the interactions between individual agents which govern the collective behaviours of the system. In this paper we investigate the impact that the underlying network has on performance in the context of collective learning.
  arXiv  Detail & Related papers  (2021-06-01T17:39:26Z)
• A game-theoretic analysis of networked system control for common-pool resource management using multi-agent reinforcement learning [54.55119659523629]
  Multi-agent reinforcement learning has recently shown great promise as an approach to networked system control. Common-pool resources include arable land, fresh water, wetlands, wildlife, fish stock, forests and the atmosphere.
  arXiv  Detail & Related papers  (2020-10-15T14:12:26Z)
• Reputation-driven Decision-making in Networks of Stochastic Agents [0.0]
  We propose a Markov Decision Process-derived framework, called RepNet-MDP, tailored to domains in which agent reputation is a key driver of the interactions between agents. In a series of experiments, RepNet agents are shown to be able to adapt their own behavior to the past behavior and reliability of the remaining agents of the network.
  arXiv  Detail & Related papers  (2020-08-26T20:21:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.