Don't lie to your friends: Learning what you know from collaborative self-play

• URL: http://arxiv.org/abs/2503.14481v2
• Date: Mon, 31 Mar 2025 21:28:02 GMT
• Title: Don't lie to your friends: Learning what you know from collaborative self-play
• Authors: Jacob Eisenstein, Reza Aghajani, Adam Fisch, Dheeru Dua, Fantine Huot, Mirella Lapata, Vicky Zayats, Jonathan Berant,
• Abstract summary: We propose a radically new approach to teaching AI agents what they know.<n>We construct multi-agent collaborations in which the group is rewarded for collectively arriving at correct answers.<n>The desired meta-knowledge emerges from the incentives built into the structure of the interaction.
• Score: 90.35507959579331
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: To be helpful assistants, AI agents must be aware of their own capabilities and limitations. This includes knowing when to answer from parametric knowledge versus using tools, when to trust tool outputs, and when to abstain or hedge. Such capabilities are hard to teach through supervised fine-tuning because they require constructing examples that reflect the agent's specific capabilities. We therefore propose a radically new approach to teaching agents what they know: \emph{collaborative self-play}. We construct multi-agent collaborations in which the group is rewarded for collectively arriving at correct answers. The desired meta-knowledge emerges from the incentives built into the structure of the interaction. We focus on small societies of agents that have access to heterogeneous tools (corpus-specific retrieval), and therefore must collaborate to maximize their success while minimizing their effort. Experiments show that group-level rewards for multi-agent communities can induce policies that \emph{transfer} to improve tool use and selective prediction in settings where individual agents are deployed in isolation.

Related papers

• Learning to Use Tools via Cooperative and Interactive Agents [58.77710337157665]
  Tool learning empowers large language models (LLMs) as agents to use external tools and extend their utility. We propose ConAgents, a Cooperative and interactive Agents framework, which coordinates three specialized agents for tool selection, tool execution, and action calibration separately. Our experiments on three datasets show that the LLMs, when equipped with ConAgents, outperform baselines with substantial improvement.
  arXiv  Detail & Related papers  (2024-03-05T15:08:16Z)
• Peer Learning: Learning Complex Policies in Groups from Scratch via Action Recommendations [16.073203911932872]
  Peer learning is a novel high-level reinforcement learning framework for agents learning in groups. We show that peer learning is able to outperform single agent learning and the baseline in several challenging OpenAI Gym domains.
  arXiv  Detail & Related papers  (2023-12-15T17:01:35Z)
• ProAgent: Building Proactive Cooperative Agents with Large Language Models [89.53040828210945]
  ProAgent is a novel framework that harnesses large language models to create proactive agents. ProAgent can analyze the present state, and infer the intentions of teammates from observations. ProAgent exhibits a high degree of modularity and interpretability, making it easily integrated into various coordination scenarios.
  arXiv  Detail & Related papers  (2023-08-22T10:36:56Z)
• AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors [93.38830440346783]
  We propose a multi-agent framework framework that can collaboratively adjust its composition as a greater-than-the-sum-of-its-parts system. Our experiments demonstrate that framework framework can effectively deploy multi-agent groups that outperform a single agent. In view of these behaviors, we discuss some possible strategies to leverage positive ones and mitigate negative ones for improving the collaborative potential of multi-agent groups.
  arXiv  Detail & Related papers  (2023-08-21T16:47:11Z)
• Learning to Learn Group Alignment: A Self-Tuning Credo Framework with Multiagent Teams [1.370633147306388]
  Mixed incentives among a population with multiagent teams has been shown to have advantages over a fully cooperative system. We propose a framework where individual learning agents self-regulate their configuration of incentives through various parts of their reward function.
  arXiv  Detail & Related papers  (2023-04-14T18:16:19Z)
• ELIGN: Expectation Alignment as a Multi-Agent Intrinsic Reward [29.737986509769808]
  We propose a self-supervised intrinsic reward ELIGN - expectation alignment. Similar to how animals collaborate in a decentralized manner with those in their vicinity, agents trained with expectation alignment learn behaviors that match their neighbors' expectations. We show that agent coordination improves through expectation alignment because agents learn to divide tasks amongst themselves, break coordination symmetries, and confuse adversaries.
  arXiv  Detail & Related papers  (2022-10-09T22:24:44Z)
• Multi-agent Deep Covering Skill Discovery [50.812414209206054]
  We propose Multi-agent Deep Covering Option Discovery, which constructs the multi-agent options through minimizing the expected cover time of the multiple agents' joint state space. Also, we propose a novel framework to adopt the multi-agent options in the MARL process. We show that the proposed algorithm can effectively capture the agent interactions with the attention mechanism, successfully identify multi-agent options, and significantly outperforms prior works using single-agent options or no options.
  arXiv  Detail & Related papers  (2022-10-07T00:40:59Z)
• Learning to Cooperate with Unseen Agent via Meta-Reinforcement Learning [4.060731229044571]
  Ad hoc teamwork problem describes situations where an agent has to cooperate with previously unseen agents to achieve a common goal. One could implement cooperative skills into an agent by using domain knowledge to design the agent's behavior. We apply meta-reinforcement learning (meta-RL) formulation in the context of the ad hoc teamwork problem.
  arXiv  Detail & Related papers  (2021-11-05T12:01:28Z)
• BGC: Multi-Agent Group Belief with Graph Clustering [1.9949730506194252]
  We propose a semi-communication method to enable agents can exchange information without communication. Inspired by the neighborhood cognitive consistency, we propose a group-based module to divide adjacent agents into a small group and minimize in-group agents' beliefs. Results reveal that the proposed method achieves a significant improvement in the SMAC benchmark.
  arXiv  Detail & Related papers  (2020-08-20T07:07:20Z)
• Learning to Incentivize Other Learning Agents [73.03133692589532]
  We show how to equip RL agents with the ability to give rewards directly to other agents, using a learned incentive function. Such agents significantly outperform standard RL and opponent-shaping agents in challenging general-sum Markov games. Our work points toward more opportunities and challenges along the path to ensure the common good in a multi-agent future.
  arXiv  Detail & Related papers  (2020-06-10T20:12:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.