Inferring Latent Temporal Sparse Coordination Graph for Multi-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2403.19253v1
• Date: Thu, 28 Mar 2024 09:20:15 GMT
• Title: Inferring Latent Temporal Sparse Coordination Graph for Multi-Agent Reinforcement Learning
• Authors: Wei Duan, Jie Lu, Junyu Xuan,
• Abstract summary: We propose a Latent Temporal Sparse Coordination Graph (LTS-CG) for Multi-Agent Reinforcement Learning (MARL) This graph learning process is further augmented by two innovative characteristics: Predict-Future, which enables agents to foresee upcoming observations, and Infer-Present, ensuring a thorough grasp of the environmental context from limited data. Our demonstrated results on the StarCraft II benchmark underscore LTS-CG's superior performance.
• Score: 19.386588137176933
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Effective agent coordination is crucial in cooperative Multi-Agent Reinforcement Learning (MARL). While agent cooperation can be represented by graph structures, prevailing graph learning methods in MARL are limited. They rely solely on one-step observations, neglecting crucial historical experiences, leading to deficient graphs that foster redundant or detrimental information exchanges. Additionally, high computational demands for action-pair calculations in dense graphs impede scalability. To address these challenges, we propose inferring a Latent Temporal Sparse Coordination Graph (LTS-CG) for MARL. The LTS-CG leverages agents' historical observations to calculate an agent-pair probability matrix, where a sparse graph is sampled from and used for knowledge exchange between agents, thereby simultaneously capturing agent dependencies and relation uncertainty. The computational complexity of this procedure is only related to the number of agents. This graph learning process is further augmented by two innovative characteristics: Predict-Future, which enables agents to foresee upcoming observations, and Infer-Present, ensuring a thorough grasp of the environmental context from limited data. These features allow LTS-CG to construct temporal graphs from historical and real-time information, promoting knowledge exchange during policy learning and effective collaboration. Graph learning and agent training occur simultaneously in an end-to-end manner. Our demonstrated results on the StarCraft II benchmark underscore LTS-CG's superior performance.

Related papers

• Can Graph Learning Improve Planning in LLM-based Agents? [61.47027387839096]
  Task planning in language agents is emerging as an important research topic alongside the development of large language models (LLMs) In this paper, we explore graph learning-based methods for task planning, a direction that is to the prevalent focus on prompt design. Our interest in graph learning stems from a theoretical discovery: the biases of attention and auto-regressive loss impede LLMs' ability to effectively navigate decision-making on graphs.
  arXiv  Detail & Related papers  (2024-05-29T14:26:24Z)
• Group-Aware Coordination Graph for Multi-Agent Reinforcement Learning [19.386588137176933]
  Group-Aware Coordination Graph (GACG) is designed to capture cooperation between agent pairs based on current observations. GACG is further used in graph convolution for information exchange between agents during decision-making. Our evaluations, conducted on StarCraft II micromanagement tasks, demonstrate GACG's superior performance.
  arXiv  Detail & Related papers  (2024-04-17T01:17:10Z)
• K-Link: Knowledge-Link Graph from LLMs for Enhanced Representation Learning in Multivariate Time-Series Data [39.83677994033754]
  We propose a novel framework named K-Link, leveraging Large Language Models (LLMs) to encode extensive general knowledge. We propose a graph alignment module, facilitating the transfer of semantic knowledge within the knowledge-link graph into the MTS-derived graph.
  arXiv  Detail & Related papers  (2024-03-06T12:08:14Z)
• MTRGL:Effective Temporal Correlation Discerning through Multi-modal Temporal Relational Graph Learning [2.879827542529434]
  We introduce a novel framework, Multi-modal Temporal Relation Graph Learning (MTRGL) MTRGL combines time series data and discrete features into a temporal graph and employs a memory-based temporal graph neural network. Our experiments on real-world datasets confirm the superior performance of MTRGL.
  arXiv  Detail & Related papers  (2024-01-25T14:21:14Z)
• Graph Relation Aware Continual Learning [3.908470250825618]
  Continual graph learning (CGL) studies the problem of learning from an infinite stream of graph data. We design a relation-aware adaptive model, dubbed as RAM-CG, that consists of a relation-discovery modular to explore latent relations behind edges. RAM-CG provides significant 2.2%, 6.9% and 6.6% accuracy improvements over the state-of-the-art results on CitationNet, OGBN-arxiv and TWITCH dataset.
  arXiv  Detail & Related papers  (2023-08-16T09:53:20Z)
• Co-attention Graph Pooling for Efficient Pairwise Graph Interaction Learning [19.58671020943416]
  Graph Neural Networks (GNNs) have proven to be effective in processing and learning from graph-structured data. We propose a novel and efficient graph-level approach for extracting interaction representations using co-attention in graph pooling. Our method, Co-Attention Graph Pooling (CAGPool), exhibits competitive performance relative to existing methods in both classification and regression tasks.
  arXiv  Detail & Related papers  (2023-07-28T07:53:34Z)
• Distributed Learning over Networks with Graph-Attention-Based Personalization [49.90052709285814]
  We propose a graph-based personalized algorithm (GATTA) for distributed deep learning. In particular, the personalized model in each agent is composed of a global part and a node-specific part. By treating each agent as one node in a graph the node-specific parameters as its features, the benefits of the graph attention mechanism can be inherited.
  arXiv  Detail & Related papers  (2023-05-22T13:48:30Z)
• Localized Contrastive Learning on Graphs [110.54606263711385]
  We introduce a simple yet effective contrastive model named Localized Graph Contrastive Learning (Local-GCL) In spite of its simplicity, Local-GCL achieves quite competitive performance in self-supervised node representation learning tasks on graphs with various scales and properties.
  arXiv  Detail & Related papers  (2022-12-08T23:36:00Z)
• GraphMAE: Self-Supervised Masked Graph Autoencoders [52.06140191214428]
  We present a masked graph autoencoder GraphMAE that mitigates issues for generative self-supervised graph learning. We conduct extensive experiments on 21 public datasets for three different graph learning tasks. The results manifest that GraphMAE--a simple graph autoencoder with our careful designs--can consistently generate outperformance over both contrastive and generative state-of-the-art baselines.
  arXiv  Detail & Related papers  (2022-05-22T11:57:08Z)
• Tensor Graph Convolutional Networks for Multi-relational and Robust Learning [74.05478502080658]
  This paper introduces a tensor-graph convolutional network (TGCN) for scalable semi-supervised learning (SSL) from data associated with a collection of graphs, that are represented by a tensor. The proposed architecture achieves markedly improved performance relative to standard GCNs, copes with state-of-the-art adversarial attacks, and leads to remarkable SSL performance over protein-to-protein interaction networks.
  arXiv  Detail & Related papers  (2020-03-15T02:33:21Z)
• Graph Representation Learning via Graphical Mutual Information Maximization [86.32278001019854]
  We propose a novel concept, Graphical Mutual Information (GMI), to measure the correlation between input graphs and high-level hidden representations. We develop an unsupervised learning model trained by maximizing GMI between the input and output of a graph neural encoder.
  arXiv  Detail & Related papers  (2020-02-04T08:33:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.