Effective Communications: A Joint Learning and Communication Framework for Multi-Agent Reinforcement Learning over Noisy Channels

• URL: http://arxiv.org/abs/2101.10369v2
• Date: Thu, 1 Apr 2021 17:30:45 GMT
• Title: Effective Communications: A Joint Learning and Communication Framework for Multi-Agent Reinforcement Learning over Noisy Channels
• Authors: Tze-Yang Tung, Szymon Kobus, Joan Roig Pujol, Deniz Gunduz
• Abstract summary: We propose a novel formulation of the "effectiveness problem" in communications. We consider multiple agents communicating over a noisy channel in order to achieve better coordination and cooperation. We show via examples that the joint policy learned using the proposed framework is superior to that where the communication is considered separately.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a novel formulation of the "effectiveness problem" in communications, put forth by Shannon and Weaver in their seminal work [2], by considering multiple agents communicating over a noisy channel in order to achieve better coordination and cooperation in a multi-agent reinforcement learning (MARL) framework. Specifically, we consider a multi-agent partially observable Markov decision process (MA-POMDP), in which the agents, in addition to interacting with the environment can also communicate with each other over a noisy communication channel. The noisy communication channel is considered explicitly as part of the dynamics of the environment and the message each agent sends is part of the action that the agent can take. As a result, the agents learn not only to collaborate with each other but also to communicate "effectively" over a noisy channel. This framework generalizes both the traditional communication problem, where the main goal is to convey a message reliably over a noisy channel, and the "learning to communicate" framework that has received recent attention in the MARL literature, where the underlying communication channels are assumed to be error-free. We show via examples that the joint policy learned using the proposed framework is superior to that where the communication is considered separately from the underlying MA-POMDP. This is a very powerful framework, which has many real world applications, from autonomous vehicle planning to drone swarm control, and opens up the rich toolbox of deep reinforcement learning for the design of multi-user communication systems.

Related papers

• Will 6G be Semantic Communications? Opportunities and Challenges from Task Oriented and Secure Communications to Integrated Sensing [49.83882366499547]
  This paper explores opportunities and challenges of task (goal)-oriented and semantic communications for next-generation (NextG) networks through the integration of multi-task learning. We employ deep neural networks representing a dedicated encoder at the transmitter and multiple task-specific decoders at the receiver. We scrutinize potential vulnerabilities stemming from adversarial attacks during both training and testing phases.
  arXiv  Detail & Related papers  (2024-01-03T04:01:20Z)
• Multi-Agent Reinforcement Learning Based on Representational Communication for Large-Scale Traffic Signal Control [13.844458247041711]
  Traffic signal control (TSC) is a challenging problem within intelligent transportation systems. We propose a communication-based MARL framework for large-scale TSC. Our framework allows each agent to learn a communication policy that dictates "which" part of the message is sent "to whom"
  arXiv  Detail & Related papers  (2023-10-03T21:06:51Z)
• Over-communicate no more: Situated RL agents learn concise communication protocols [78.28898217947467]
  It is unclear how to design artificial agents that can learn to effectively and efficiently communicate with each other. Much research on communication emergence uses reinforcement learning (RL) We explore situated communication in a multi-step task, where the acting agent has to forgo an environmental action to communicate. We find that while all tested pressures can disincentivise over-communication, situated communication does it most effectively and, unlike the cost on effort, does not negatively impact emergence.
  arXiv  Detail & Related papers  (2022-11-02T21:08:14Z)
• Coordinating Policies Among Multiple Agents via an Intelligent Communication Channel [81.39444892747512]
  In Multi-Agent Reinforcement Learning (MARL), specialized channels are often introduced that allow agents to communicate directly with one another. We propose an alternative approach whereby agents communicate through an intelligent facilitator that learns to sift through and interpret signals provided by all agents to improve the agents' collective performance.
  arXiv  Detail & Related papers  (2022-05-21T14:11:33Z)
• Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agents [83.52684405389445]
  We introduce the collaborative multi-object navigation task CoMON. In this task, an oracle agent has detailed environment information in the form of a map. It communicates with a navigator agent that perceives the environment visually and is tasked to find a sequence of goals. We show that the emergent communication can be grounded to the agent observations and the spatial structure of the 3D environment.
  arXiv  Detail & Related papers  (2021-10-12T06:56:11Z)
• Learning Emergent Discrete Message Communication for Cooperative Reinforcement Learning [36.468498804251574]
  We show that discrete message communication has performance comparable to continuous message communication. We propose an approach that allows humans to interactively send discrete messages to agents.
  arXiv  Detail & Related papers  (2021-02-24T20:44:14Z)
• Exploring Zero-Shot Emergent Communication in Embodied Multi-Agent Populations [59.608216900601384]
  We study agents that learn to communicate via actuating their joints in a 3D environment. We show that under realistic assumptions, a non-uniform distribution of intents and a common-knowledge energy cost, these agents can find protocols that generalize to novel partners.
  arXiv  Detail & Related papers  (2020-10-29T19:23:10Z)
• The Emergence of Adversarial Communication in Multi-Agent Reinforcement Learning [6.18778092044887]
  Many real-world problems require the coordination of multiple autonomous agents. Recent work has shown the promise of Graph Neural Networks (GNNs) to learn explicit communication strategies that enable complex multi-agent coordination. We show how a single self-interested agent is capable of learning highly manipulative communication strategies that allows it to significantly outperform a cooperative team of agents.
  arXiv  Detail & Related papers  (2020-08-06T12:48:08Z)
• Learning to Communicate Using Counterfactual Reasoning [2.8110705488739676]
  This paper introduces the novel multi-agent counterfactual communication learning (MACC) method. MACC adapts counterfactual reasoning in order to overcome the credit assignment problem for communicating agents. Our experiments show that MACC is able to outperform the state-of-the-art baselines in four different scenarios in the Particle environment.
  arXiv  Detail & Related papers  (2020-06-12T14:02:04Z)
• Learning Individually Inferred Communication for Multi-Agent Cooperation [37.56115000150748]
  We propose Individually Inferred Communication (I2C) to enable agents to learn a prior for agent-agent communication. The prior knowledge is learned via causal inference and realized by a feed-forward neural network. I2C can not only reduce communication overhead but also improve the performance in a variety of multi-agent cooperative scenarios.
  arXiv  Detail & Related papers  (2020-06-11T14:07:57Z)
• Learning Structured Communication for Multi-agent Reinforcement Learning [104.64584573546524]
  This work explores the large-scale multi-agent communication mechanism under a multi-agent reinforcement learning (MARL) setting. We propose a novel framework termed as Learning Structured Communication (LSC) by using a more flexible and efficient communication topology.
  arXiv  Detail & Related papers  (2020-02-11T07:19:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.