Learning to Communicate Using Counterfactual Reasoning
- URL: http://arxiv.org/abs/2006.07200v4
- Date: Tue, 26 Apr 2022 10:05:33 GMT
- Title: Learning to Communicate Using Counterfactual Reasoning
- Authors: Simon Vanneste, Astrid Vanneste, Kevin Mets, Tom De Schepper, Ali
Anwar, Siegfried Mercelis, Steven Latr\'e, Peter Hellinckx
- Abstract summary: This paper introduces the novel multi-agent counterfactual communication learning (MACC) method.
MACC adapts counterfactual reasoning in order to overcome the credit assignment problem for communicating agents.
Our experiments show that MACC is able to outperform the state-of-the-art baselines in four different scenarios in the Particle environment.
- Score: 2.8110705488739676
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Learning to communicate in order to share state information is an active
problem in the area of multi-agent reinforcement learning (MARL). The credit
assignment problem, the non-stationarity of the communication environment and
the creation of influenceable agents are major challenges within this research
field which need to be overcome in order to learn a valid communication
protocol. This paper introduces the novel multi-agent counterfactual
communication learning (MACC) method which adapts counterfactual reasoning in
order to overcome the credit assignment problem for communicating agents.
Secondly, the non-stationarity of the communication environment while learning
the communication Q-function is overcome by creating the communication
Q-function using the action policy of the other agents and the Q-function of
the action environment. Additionally, a social loss function is introduced in
order to create influenceable agents which is required to learn a valid
communication protocol. Our experiments show that MACC is able to outperform
the state-of-the-art baselines in four different scenarios in the Particle
environment.
Related papers
- Hypergame Theory for Decentralized Resource Allocation in Multi-user Semantic Communications [60.63472821600567]
A novel framework for decentralized computing and communication resource allocation in multiuser SC systems is proposed.
The challenge of efficiently allocating communication and computing resources is addressed through the application of Stackelberg hyper game theory.
Simulation results show that the proposed Stackelberg hyper game results in efficient usage of communication and computing resources.
arXiv Detail & Related papers (2024-09-26T15:55:59Z) - CAMEL: Communicative Agents for "Mind" Exploration of Large Language
Model Society [58.04479313658851]
This paper explores the potential of building scalable techniques to facilitate autonomous cooperation among communicative agents.
We propose a novel communicative agent framework named role-playing.
Our contributions include introducing a novel communicative agent framework, offering a scalable approach for studying the cooperative behaviors and capabilities of multi-agent systems.
arXiv Detail & Related papers (2023-03-31T01:09:00Z) - AC2C: Adaptively Controlled Two-Hop Communication for Multi-Agent
Reinforcement Learning [4.884877440051105]
We propose a novel communication protocol called Adaptively Controlled Two-Hop Communication (AC2C)
AC2C employs an adaptive two-hop communication strategy to enable long-range information exchange among agents to boost performance.
We evaluate AC2C on three cooperative multi-agent tasks, and the experimental results show that it outperforms relevant baselines with lower communication costs.
arXiv Detail & Related papers (2023-02-24T09:00:34Z) - Over-communicate no more: Situated RL agents learn concise communication
protocols [78.28898217947467]
It is unclear how to design artificial agents that can learn to effectively and efficiently communicate with each other.
Much research on communication emergence uses reinforcement learning (RL)
We explore situated communication in a multi-step task, where the acting agent has to forgo an environmental action to communicate.
We find that while all tested pressures can disincentivise over-communication, situated communication does it most effectively and, unlike the cost on effort, does not negatively impact emergence.
arXiv Detail & Related papers (2022-11-02T21:08:14Z) - Interpretation of Emergent Communication in Heterogeneous Collaborative
Embodied Agents [83.52684405389445]
We introduce the collaborative multi-object navigation task CoMON.
In this task, an oracle agent has detailed environment information in the form of a map.
It communicates with a navigator agent that perceives the environment visually and is tasked to find a sequence of goals.
We show that the emergent communication can be grounded to the agent observations and the spatial structure of the 3D environment.
arXiv Detail & Related papers (2021-10-12T06:56:11Z) - Learning Selective Communication for Multi-Agent Path Finding [18.703918339797283]
Decision Causal Communication (DCC) is a simple yet efficient model to enable agents to select neighbors to conduct communication.
DCC is suitable for decentralized execution to handle large scale problems.
arXiv Detail & Related papers (2021-09-12T03:07:20Z) - Effective Communications: A Joint Learning and Communication Framework
for Multi-Agent Reinforcement Learning over Noisy Channels [0.0]
We propose a novel formulation of the "effectiveness problem" in communications.
We consider multiple agents communicating over a noisy channel in order to achieve better coordination and cooperation.
We show via examples that the joint policy learned using the proposed framework is superior to that where the communication is considered separately.
arXiv Detail & Related papers (2021-01-02T10:43:41Z) - Exploring Zero-Shot Emergent Communication in Embodied Multi-Agent
Populations [59.608216900601384]
We study agents that learn to communicate via actuating their joints in a 3D environment.
We show that under realistic assumptions, a non-uniform distribution of intents and a common-knowledge energy cost, these agents can find protocols that generalize to novel partners.
arXiv Detail & Related papers (2020-10-29T19:23:10Z) - Learning Individually Inferred Communication for Multi-Agent Cooperation [37.56115000150748]
We propose Individually Inferred Communication (I2C) to enable agents to learn a prior for agent-agent communication.
The prior knowledge is learned via causal inference and realized by a feed-forward neural network.
I2C can not only reduce communication overhead but also improve the performance in a variety of multi-agent cooperative scenarios.
arXiv Detail & Related papers (2020-06-11T14:07:57Z) - Learning Structured Communication for Multi-agent Reinforcement Learning [104.64584573546524]
This work explores the large-scale multi-agent communication mechanism under a multi-agent reinforcement learning (MARL) setting.
We propose a novel framework termed as Learning Structured Communication (LSC) by using a more flexible and efficient communication topology.
arXiv Detail & Related papers (2020-02-11T07:19:45Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.