Dynamic population-based meta-learning for multi-agent communication with natural language

• URL: http://arxiv.org/abs/2110.14241v1
• Date: Wed, 27 Oct 2021 07:50:02 GMT
• Title: Dynamic population-based meta-learning for multi-agent communication with natural language
• Authors: Abhinav Gupta, Marc Lanctot, Angeliki Lazaridou
• Abstract summary: We train agents that can coordinate with seen, unseen as well as human partners in a multi-agent communication environment involving natural language. We propose a dynamic population-based meta-learning approach that builds such a population in an iterative manner. We show that our agents outperform all prior work when communicating with seen partners and humans.
• Score: 44.87604064505434
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work, our goal is to train agents that can coordinate with seen, unseen as well as human partners in a multi-agent communication environment involving natural language. Previous work using a single set of agents has shown great progress in generalizing to known partners, however it struggles when coordinating with unfamiliar agents. To mitigate that, recent work explored the use of population-based approaches, where multiple agents interact with each other with the goal of learning more generic protocols. These methods, while able to result in good coordination between unseen partners, still only achieve so in cases of simple languages, thus failing to adapt to human partners using natural language. We attribute this to the use of static populations and instead propose a dynamic population-based meta-learning approach that builds such a population in an iterative manner. We perform a holistic evaluation of our method on two different referential games, and show that our agents outperform all prior work when communicating with seen partners and humans. Furthermore, we analyze the natural language generation skills of our agents, where we find that our agents also outperform strong baselines. Finally, we test the robustness of our agents when communicating with out-of-population agents and carefully test the importance of each component of our method through ablation studies.

Related papers

• COMMA: A Communicative Multimodal Multi-Agent Benchmark [7.831385481814481]
  We introduce a novel benchmark designed to evaluate the collaborative performance of multimodal multi-agent systems through language communication. By testing both agent-agent and agent-human collaborations using open-source and closed-source models, our findings reveal surprising weaknesses in state-of-the-art models.
  arXiv  Detail & Related papers  (2024-10-10T02:49:47Z)
• Bidirectional Emergent Language in Situated Environments [4.950411915351642]
  We introduce two novel cooperative environments: Multi-Agent Pong and Collectors. optimal performance requires the emergence of a communication protocol, but moderate success can be achieved without one. We find that the emerging communication is sparse, with the agents only generating meaningful messages and acting upon incoming messages in states where they cannot succeed without coordination.
  arXiv  Detail & Related papers  (2024-08-26T21:25:44Z)
• PersLLM: A Personified Training Approach for Large Language Models [66.16513246245401]
  We propose PersLLM, integrating psychology-grounded principles of personality: social practice, consistency, and dynamic development. We incorporate personality traits directly into the model parameters, enhancing the model's resistance to induction, promoting consistency, and supporting the dynamic evolution of personality.
  arXiv  Detail & Related papers  (2024-07-17T08:13:22Z)
• Enhancing Multi-Agent Coordination through Common Operating Picture Integration [14.927199437011044]
  We present an approach to multi-agent coordination, where each agent is equipped with the capability to integrate its history of observations, actions and messages received into a Common Operating Picture (COP) Our results demonstrate the efficacy of COP integration, and show that COP-based training leads to robust policies compared to state-of-the-art Multi-Agent Reinforcement Learning (MARL) methods when faced with out-of-distribution initial states.
  arXiv  Detail & Related papers  (2023-11-08T15:08:55Z)
• ProAgent: Building Proactive Cooperative Agents with Large Language Models [89.53040828210945]
  ProAgent is a novel framework that harnesses large language models to create proactive agents. ProAgent can analyze the present state, and infer the intentions of teammates from observations. ProAgent exhibits a high degree of modularity and interpretability, making it easily integrated into various coordination scenarios.
  arXiv  Detail & Related papers  (2023-08-22T10:36:56Z)
• Building Cooperative Embodied Agents Modularly with Large Language Models [104.57849816689559]
  We address challenging multi-agent cooperation problems with decentralized control, raw sensory observations, costly communication, and multi-objective tasks instantiated in various embodied environments. We harness the commonsense knowledge, reasoning ability, language comprehension, and text generation prowess of LLMs and seamlessly incorporate them into a cognitive-inspired modular framework. Our experiments on C-WAH and TDW-MAT demonstrate that CoELA driven by GPT-4 can surpass strong planning-based methods and exhibit emergent effective communication.
  arXiv  Detail & Related papers  (2023-07-05T17:59:27Z)
• Toward Collaborative Reinforcement Learning Agents that Communicate Through Text-Based Natural Language [4.289574109162585]
  This paper considers text-based natural language as a novel form of communication between agents trained with reinforcement learning. Inspired by the game of Blind Leads, we propose an environment where one agent uses natural language instructions to guide another through a maze.
  arXiv  Detail & Related papers  (2021-07-20T09:19:29Z)
• Multi-agent Communication meets Natural Language: Synergies between Functional and Structural Language Learning [16.776753238108036]
  We present a method for combining multi-agent communication and traditional data-driven approaches to natural language learning. Our starting point is a language model that has been trained on generic, not task-specific language data. We then place this model in a multi-agent self-play environment that generates task-specific rewards used to adapt or modulate the model.
  arXiv  Detail & Related papers  (2020-05-14T15:32:23Z)
• Learning to cooperate: Emergent communication in multi-agent navigation [49.11609702016523]
  We show that agents performing a cooperative navigation task learn an interpretable communication protocol. An analysis of the agents' policies reveals that emergent signals spatially cluster the state space. Using populations of agents, we show that the emergent protocol has basic compositional structure.
  arXiv  Detail & Related papers  (2020-04-02T16:03:17Z)
• On the interaction between supervision and self-play in emergent communication [82.290338507106]
  We investigate the relationship between two categories of learning signals with the ultimate goal of improving sample efficiency. We find that first training agents via supervised learning on human data followed by self-play outperforms the converse.
  arXiv  Detail & Related papers  (2020-02-04T02:35:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.