A Decentralized Communication Framework based on Dual-Level Recurrence for Multi-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2202.10612v1
• Date: Tue, 22 Feb 2022 01:36:59 GMT
• Title: A Decentralized Communication Framework based on Dual-Level Recurrence for Multi-Agent Reinforcement Learning
• Authors: Jingchen Li and Haobin Shi and Kao-Shing Hwang
• Abstract summary: We present a dual-level recurrent communication framework for multi-agent systems. The first recurrence occurs in the communication sequence and is used to transmit communication data among agents. The second recurrence is based on the time sequence and combines the historical observations for each agent.
• Score: 5.220940151628735
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a model enabling decentralized multiple agents to share their perception of environment in a fair and adaptive way. In our model, both the current message and historical observation are taken into account, and they are handled in the same recurrent model but in different forms. We present a dual-level recurrent communication framework for multi-agent systems, in which the first recurrence occurs in the communication sequence and is used to transmit communication data among agents, while the second recurrence is based on the time sequence and combines the historical observations for each agent. The developed communication flow separates communication messages from memories but allows agents to share their historical observations by the dual-level recurrence. This design makes agents adapt to changeable communication objects, while the communication results are fair to these agents. We provide a sufficient discussion about our method in both partially observable and fully observable environments. The results of several experiments suggest our method outperforms the existing decentralized communication frameworks and the corresponding centralized training method.

Related papers

• Communication Learning in Multi-Agent Systems from Graph Modeling Perspective [62.13508281188895]
  We introduce a novel approach wherein we conceptualize the communication architecture among agents as a learnable graph. We introduce a temporal gating mechanism for each agent, enabling dynamic decisions on whether to receive shared information at a given time.
  arXiv  Detail & Related papers  (2024-11-01T05:56:51Z)
• Semantic Communication for Cooperative Perception using HARQ [51.148203799109304]
  We leverage an importance map to distill critical semantic information, introducing a cooperative perception semantic communication framework. To counter the challenges posed by time-varying multipath fading, our approach incorporates the use of frequency-division multiplexing (OFDM) along with channel estimation and equalization strategies. We introduce a novel semantic error detection method that is integrated with our semantic communication framework in the spirit of hybrid automatic repeated request (HARQ)
  arXiv  Detail & Related papers  (2024-08-29T08:53:26Z)
• Learning Multi-Agent Communication from Graph Modeling Perspective [62.13508281188895]
  We introduce a novel approach wherein we conceptualize the communication architecture among agents as a learnable graph. Our proposed approach, CommFormer, efficiently optimize the communication graph and concurrently refines architectural parameters through gradient descent in an end-to-end manner.
  arXiv  Detail & Related papers  (2024-05-14T12:40:25Z)
• T2MAC: Targeted and Trusted Multi-Agent Communication through Selective Engagement and Evidence-Driven Integration [15.91335141803629]
  We propose Targeted and Trusted Multi-Agent Communication (T2MAC) to help agents learn selective engagement and evidence-driven integration. T2MAC enables agents to craft individualized messages, pinpoint ideal communication windows, and engage with reliable partners. We evaluate our method on a diverse set of cooperative multi-agent tasks, with varying difficulties, involving different scales.
  arXiv  Detail & Related papers  (2024-01-19T18:00:33Z)
• Centralized Training with Hybrid Execution in Multi-Agent Reinforcement Learning [7.163485179361718]
  We introduce hybrid execution in multi-agent reinforcement learning (MARL) MARL is a new paradigm in which agents aim to successfully complete cooperative tasks with arbitrary communication levels at execution time. We contribute MARO, an approach that makes use of an auto-regressive predictive model, trained in a centralized manner, to estimate missing agents' observations.
  arXiv  Detail & Related papers  (2022-10-12T14:58:32Z)
• Multi-Agent Coordination via Multi-Level Communication [29.388570369796586]
  We propose a novel multi-level communication scheme, Sequential Communication (SeqComm) In this paper, we propose a novel multi-level communication scheme, Sequential Communication (SeqComm)
  arXiv  Detail & Related papers  (2022-09-26T14:08:03Z)
• Multi-Agent Adversarial Attacks for Multi-Channel Communications [24.576538640840976]
  We propose a multi-agent adversary system (MAAS) for modeling and analyzing adversaries in a wireless communication scenario. By modeling the adversaries as learning agents, we show that the proposed MAAS is able to successfully choose the transmitted channel(s) and their respective allocated power(s) without any prior knowledge of the sender strategy.
  arXiv  Detail & Related papers  (2022-01-22T23:57:00Z)
• Multi-Agent Imitation Learning with Copulas [102.27052968901894]
  Multi-agent imitation learning aims to train multiple agents to perform tasks from demonstrations by learning a mapping between observations and actions. In this paper, we propose to use copula, a powerful statistical tool for capturing dependence among random variables, to explicitly model the correlation and coordination in multi-agent systems. Our proposed model is able to separately learn marginals that capture the local behavioral patterns of each individual agent, as well as a copula function that solely and fully captures the dependence structure among agents.
  arXiv  Detail & Related papers  (2021-07-10T03:49:41Z)
• Inference-Based Deterministic Messaging For Multi-Agent Communication [1.8275108630751844]
  We study learning in matrix-based signaling games to show that decentralized methods can converge to a suboptimal policy. We then propose a modification to the messaging policy, in which the sender deterministically chooses the best message that helps the receiver to infer the sender's observation.
  arXiv  Detail & Related papers  (2021-03-03T03:09:22Z)
• Exploring Zero-Shot Emergent Communication in Embodied Multi-Agent Populations [59.608216900601384]
  We study agents that learn to communicate via actuating their joints in a 3D environment. We show that under realistic assumptions, a non-uniform distribution of intents and a common-knowledge energy cost, these agents can find protocols that generalize to novel partners.
  arXiv  Detail & Related papers  (2020-10-29T19:23:10Z)
• Learning Structured Communication for Multi-agent Reinforcement Learning [104.64584573546524]
  This work explores the large-scale multi-agent communication mechanism under a multi-agent reinforcement learning (MARL) setting. We propose a novel framework termed as Learning Structured Communication (LSC) by using a more flexible and efficient communication topology.
  arXiv  Detail & Related papers  (2020-02-11T07:19:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.