Multiagent Multimodal Categorization for Symbol Emergence: Emergent
Communication via Interpersonal Cross-modal Inference
- URL: http://arxiv.org/abs/2109.07194v1
- Date: Wed, 15 Sep 2021 10:20:54 GMT
- Title: Multiagent Multimodal Categorization for Symbol Emergence: Emergent
Communication via Interpersonal Cross-modal Inference
- Authors: Yoshinobu Hagiwara, Kazuma Furukawa, Akira Taniguchi, and Tadahiro
Taniguchi
- Abstract summary: This paper describes a computational model of multiagent multimodal categorization that realizes emergent communication.
Inter-MDM enables agents to form multimodal categories and appropriately share signs between agents.
It is shown that emergent communication improves categorization accuracy, even when some sensory modalities are missing.
- Score: 4.964816143841663
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: This paper describes a computational model of multiagent multimodal
categorization that realizes emergent communication. We clarify whether the
computational model can reproduce the following functions in a symbol emergence
system, comprising two agents with different sensory modalities playing a
naming game. (1) Function for forming a shared lexical system that comprises
perceptual categories and corresponding signs, formed by agents through
individual learning and semiotic communication between agents. (2) Function to
improve the categorization accuracy in an agent via semiotic communication with
another agent, even when some sensory modalities of each agent are missing. (3)
Function that an agent infers unobserved sensory information based on a sign
sampled from another agent in the same manner as cross-modal inference. We
propose an interpersonal multimodal Dirichlet mixture (Inter-MDM), which is
derived by dividing an integrative probabilistic generative model, which is
obtained by integrating two Dirichlet mixtures (DMs). The Markov chain Monte
Carlo algorithm realizes emergent communication. The experimental results
demonstrated that Inter-MDM enables agents to form multimodal categories and
appropriately share signs between agents. It is shown that emergent
communication improves categorization accuracy, even when some sensory
modalities are missing. Inter-MDM enables an agent to predict unobserved
information based on a shared sign.
Related papers
- Scaling Large-Language-Model-based Multi-Agent Collaboration [75.5241464256688]
Pioneering advancements in large language model-powered agents have underscored the design pattern of multi-agent collaboration.
Inspired by the neural scaling law, this study investigates whether a similar principle applies to increasing agents in multi-agent collaboration.
arXiv Detail & Related papers (2024-06-11T11:02:04Z) - SpeechAgents: Human-Communication Simulation with Multi-Modal
Multi-Agent Systems [53.94772445896213]
Large Language Model (LLM)-based multi-agent systems have demonstrated promising performance in simulating human society.
We propose SpeechAgents, a multi-modal LLM based multi-agent system designed for simulating human communication.
arXiv Detail & Related papers (2024-01-08T15:01:08Z) - Symbol emergence as interpersonal cross-situational learning: the
emergence of lexical knowledge with combinatoriality [5.350057408744861]
We present a computational model for a symbol emergence system in cognitive and developmental robotics.
Our proposed model facilitates the emergence of lexical knowledge with Metropolisity by performing category formation.
Our results indicate that the lexical knowledge developed using our proposed model exhibits performance for novel situations.
arXiv Detail & Related papers (2023-06-27T23:55:33Z) - MADiff: Offline Multi-agent Learning with Diffusion Models [79.18130544233794]
MADiff is a diffusion-based multi-agent learning framework.
It works as both a decentralized policy and a centralized controller.
Our experiments demonstrate that MADiff outperforms baseline algorithms across various multi-agent learning tasks.
arXiv Detail & Related papers (2023-05-27T02:14:09Z) - Unlimited Neighborhood Interaction for Heterogeneous Trajectory
Prediction [97.40338982628094]
We propose a simple yet effective Unlimited Neighborhood Interaction Network (UNIN) which predicts trajectories of heterogeneous agents in multiply categories.
Specifically, the proposed unlimited neighborhood interaction module generates the fused-features of all agents involved in an interaction simultaneously.
A hierarchical graph attention module is proposed to obtain category-tocategory interaction and agent-to-agent interaction.
arXiv Detail & Related papers (2021-07-31T13:36:04Z) - Multi-Agent Imitation Learning with Copulas [102.27052968901894]
Multi-agent imitation learning aims to train multiple agents to perform tasks from demonstrations by learning a mapping between observations and actions.
In this paper, we propose to use copula, a powerful statistical tool for capturing dependence among random variables, to explicitly model the correlation and coordination in multi-agent systems.
Our proposed model is able to separately learn marginals that capture the local behavioral patterns of each individual agent, as well as a copula function that solely and fully captures the dependence structure among agents.
arXiv Detail & Related papers (2021-07-10T03:49:41Z) - Specializing Inter-Agent Communication in Heterogeneous Multi-Agent
Reinforcement Learning using Agent Class Information [1.713291434132985]
This work proposes the representation of multi-agent communication capabilities as a directed labeled heterogeneous agent graph.
We also introduce a neural network architecture that specializes communication in fully cooperative heterogeneous multi-agent tasks.
arXiv Detail & Related papers (2020-12-14T15:09:57Z) - BGC: Multi-Agent Group Belief with Graph Clustering [1.9949730506194252]
We propose a semi-communication method to enable agents can exchange information without communication.
Inspired by the neighborhood cognitive consistency, we propose a group-based module to divide adjacent agents into a small group and minimize in-group agents' beliefs.
Results reveal that the proposed method achieves a significant improvement in the SMAC benchmark.
arXiv Detail & Related papers (2020-08-20T07:07:20Z) - The Emergence of Adversarial Communication in Multi-Agent Reinforcement
Learning [6.18778092044887]
Many real-world problems require the coordination of multiple autonomous agents.
Recent work has shown the promise of Graph Neural Networks (GNNs) to learn explicit communication strategies that enable complex multi-agent coordination.
We show how a single self-interested agent is capable of learning highly manipulative communication strategies that allows it to significantly outperform a cooperative team of agents.
arXiv Detail & Related papers (2020-08-06T12:48:08Z) - Counterfactual Multi-Agent Reinforcement Learning with Graph Convolution
Communication [5.5438676149999075]
We consider a fully cooperative multi-agent system where agents cooperate to maximize a system's utility.
We propose that multi-agent systems must have the ability to communicate and understand the inter-plays between agents.
We develop an architecture that allows for communication among agents and tailors the system's reward for each individual agent.
arXiv Detail & Related papers (2020-04-01T14:36:13Z) - Multi-Agent Interactions Modeling with Correlated Policies [53.38338964628494]
In this paper, we cast the multi-agent interactions modeling problem into a multi-agent imitation learning framework.
We develop a Decentralized Adrial Imitation Learning algorithm with Correlated policies (CoDAIL)
Various experiments demonstrate that CoDAIL can better regenerate complex interactions close to the demonstrators.
arXiv Detail & Related papers (2020-01-04T17:31:53Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.