Group-Aware Coordination Graph for Multi-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2404.10976v3
• Date: Sat, 11 May 2024 23:50:21 GMT
• Title: Group-Aware Coordination Graph for Multi-Agent Reinforcement Learning
• Authors: Wei Duan, Jie Lu, Junyu Xuan,
• Abstract summary: Group-Aware Coordination Graph (GACG) is designed to capture cooperation between agent pairs based on current observations. GACG is further used in graph convolution for information exchange between agents during decision-making. Our evaluations, conducted on StarCraft II micromanagement tasks, demonstrate GACG's superior performance.
• Score: 19.386588137176933
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cooperative Multi-Agent Reinforcement Learning (MARL) necessitates seamless collaboration among agents, often represented by an underlying relation graph. Existing methods for learning this graph primarily focus on agent-pair relations, neglecting higher-order relationships. While several approaches attempt to extend cooperation modelling to encompass behaviour similarities within groups, they commonly fall short in concurrently learning the latent graph, thereby constraining the information exchange among partially observed agents. To overcome these limitations, we present a novel approach to infer the Group-Aware Coordination Graph (GACG), which is designed to capture both the cooperation between agent pairs based on current observations and group-level dependencies from behaviour patterns observed across trajectories. This graph is further used in graph convolution for information exchange between agents during decision-making. To further ensure behavioural consistency among agents within the same group, we introduce a group distance loss, which promotes group cohesion and encourages specialization between groups. Our evaluations, conducted on StarCraft II micromanagement tasks, demonstrate GACG's superior performance. An ablation study further provides experimental evidence of the effectiveness of each component of our method.

Related papers

• Enhancing Interaction Modeling with Agent Selection and Physical Coefficient for Trajectory Prediction [1.6954753390775528]
  We present ASPILin, which manually selects interacting agents and calculates their correlations instead of attention scores. Remarkably, experiments conducted on the INTERACTION, highD, and CitySim datasets demonstrate that our method is efficient and straightforward.
  arXiv  Detail & Related papers  (2024-05-21T18:45:18Z)
• Learning Multi-Agent Communication from Graph Modeling Perspective [62.13508281188895]
  We introduce a novel approach wherein we conceptualize the communication architecture among agents as a learnable graph. Our proposed approach, CommFormer, efficiently optimize the communication graph and concurrently refines architectural parameters through gradient descent in an end-to-end manner.
  arXiv  Detail & Related papers  (2024-05-14T12:40:25Z)
• Enhancing Cooperation through Selective Interaction and Long-term Experiences in Multi-Agent Reinforcement Learning [10.932974027102619]
  This study introduces a computational framework based on multi-agent reinforcement learning in the spatial Prisoner's Dilemma game. By modelling each agent using two distinct Q-networks, we disentangle the coevolutionary dynamics between cooperation and interaction.
  arXiv  Detail & Related papers  (2024-05-04T12:42:55Z)
• Inferring Latent Temporal Sparse Coordination Graph for Multi-Agent Reinforcement Learning [19.386588137176933]
  We propose a Latent Temporal Sparse Coordination Graph (LTS-CG) for Multi-Agent Reinforcement Learning (MARL) This graph learning process is further augmented by two innovative characteristics: Predict-Future, which enables agents to foresee upcoming observations, and Infer-Present, ensuring a thorough grasp of the environmental context from limited data. Our demonstrated results on the StarCraft II benchmark underscore LTS-CG's superior performance.
  arXiv  Detail & Related papers  (2024-03-28T09:20:15Z)
• Situation-Dependent Causal Influence-Based Cooperative Multi-agent Reinforcement Learning [18.054709749075194]
  We propose a novel MARL algorithm named Situation-Dependent Causal Influence-Based Cooperative Multi-agent Reinforcement Learning (SCIC) Our approach aims to detect inter-agent causal influences in specific situations based on the criterion using causal intervention and conditional mutual information. The resulting update links coordinated exploration and intrinsic reward distribution, which enhance overall collaboration and performance.
  arXiv  Detail & Related papers  (2023-12-15T05:09:32Z)
• Learning Complete Topology-Aware Correlations Between Relations for Inductive Link Prediction [121.65152276851619]
  We show that semantic correlations between relations are inherently edge-level and entity-independent. We propose a novel subgraph-based method, namely TACO, to model Topology-Aware COrrelations between relations. To further exploit the potential of RCN, we propose Complete Common Neighbor induced subgraph.
  arXiv  Detail & Related papers  (2023-09-20T08:11:58Z)
• Decentralized Adversarial Training over Graphs [55.28669771020857]
  The vulnerability of machine learning models to adversarial attacks has been attracting considerable attention in recent years. This work studies adversarial training over graphs, where individual agents are subjected to varied strength perturbation space.
  arXiv  Detail & Related papers  (2023-03-23T15:05:16Z)
• Unrolled Graph Learning for Multi-Agent Collaboration [37.239120967721156]
  We propose a distributed multi-agent learning model inspired by human collaboration. Agents can autonomously detect suitable collaborators and refer to collaborators' model for better performance.
  arXiv  Detail & Related papers  (2022-10-31T07:05:44Z)
• ACTIVE:Augmentation-Free Graph Contrastive Learning for Partial Multi-View Clustering [52.491074276133325]
  We propose an augmentation-free graph contrastive learning framework to solve the problem of partial multi-view clustering. The proposed approach elevates instance-level contrastive learning and missing data inference to the cluster-level, effectively mitigating the impact of individual missing data on clustering.
  arXiv  Detail & Related papers  (2022-03-01T02:32:25Z)
• Multi-Agent Imitation Learning with Copulas [102.27052968901894]
  Multi-agent imitation learning aims to train multiple agents to perform tasks from demonstrations by learning a mapping between observations and actions. In this paper, we propose to use copula, a powerful statistical tool for capturing dependence among random variables, to explicitly model the correlation and coordination in multi-agent systems. Our proposed model is able to separately learn marginals that capture the local behavioral patterns of each individual agent, as well as a copula function that solely and fully captures the dependence structure among agents.
  arXiv  Detail & Related papers  (2021-07-10T03:49:41Z)
• Improving Long-Tail Relation Extraction with Collaborating Relation-Augmented Attention [63.26288066935098]
  We propose a novel neural network, Collaborating Relation-augmented Attention (CoRA), to handle both the wrong labeling and long-tail relations. In the experiments on the popular benchmark dataset NYT, the proposed CoRA improves the prior state-of-the-art performance by a large margin.
  arXiv  Detail & Related papers  (2020-10-08T05:34:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.