Learning to Negotiate via Voluntary Commitment

• URL: http://arxiv.org/abs/2503.03866v2
• Date: Wed, 19 Mar 2025 07:23:37 GMT
• Title: Learning to Negotiate via Voluntary Commitment
• Abstract summary: Partial alignment and conflict of autonomous agents lead to mixed-motive scenarios in real-world applications.<n>We propose Markov Commitment Games (MCGs), a variant of commitment games, where agents can voluntarily commit to their proposed future plans.<n> Experimental results in challenging mixed-motive tasks demonstrate faster empirical convergence and higher returns for our method compared with its counterparts.
• Score: 24.76922344331357
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The partial alignment and conflict of autonomous agents lead to mixed-motive scenarios in many real-world applications. However, agents may fail to cooperate in practice even when cooperation yields a better outcome. One well known reason for this failure comes from non-credible commitments. To facilitate commitments among agents for better cooperation, we define Markov Commitment Games (MCGs), a variant of commitment games, where agents can voluntarily commit to their proposed future plans. Based on MCGs, we propose a learnable commitment protocol via policy gradients. We further propose incentive-compatible learning to accelerate convergence to equilibria with better social welfare. Experimental results in challenging mixed-motive tasks demonstrate faster empirical convergence and higher returns for our method compared with its counterparts. Our code is available at https://github.com/shuhui-zhu/DCL.

Related papers

• Commitment To Cooperation With Self-Negotiated Contracts [3.585764648520447]
  We study the role of contract-based cooperation using LLM-based agents in CT, a spatial-temporal game that combines bargaining with navigation towards a goal.<n>We find that self-negotiated contracts can improve cooperative outcomes beyond what is possible with regular trading.
  arXiv  Detail & Related papers  (2026-07-23T11:04:58Z)
• LLawCo: Learning Laws of Cooperation for Modeling Embodied Multi-Agent Behavior [51.25229110468212]
  Embodied agents operating in decentralized environments often exhibit behaviors that are misaligned with their partners or inconsistent with the environment state.<n>We propose a novel framework, Learning Laws of Cooperation, that enables embodied agents to autonomously align with both their partners and task objectives.<n>Our framework allows agents to reflect on past failures to extract misaligned behavioral patterns, which are used to derive high-level behavioral laws, such as "Talk when necessary" and "Wait for partner"
  arXiv  Detail & Related papers  (2026-06-26T15:26:12Z)
• TRACER: Turn-level Regret Matching with Inner Reinforcement Credit for Cooperative Multi-LLM Reasoning [25.10564375661515]
  Large language models increasingly rely on either reinforcement learning or multi-agent prompting to improve reasoning.<n>We introduce TRACER, a turn-level reinforcement framework for cooperative multi-LLM reasoning.<n>We train all local RL-style methods on the GSM8K training split and evaluate on held-out GSM8K, MATH500, and GPQA-Diamond to measure in-domain accuracy, cross-benchmark generalization, inference cost, and correction-preservation behavior.
  arXiv  Detail & Related papers  (2026-05-27T16:25:21Z)
• Collaborating in Multi-Armed Bandits with Strategic Agents [37.473632043513014]
  We study collaborative learning in multi-agent Bayesian bandit problems, where strategic agents collectively solve the same bandit instance.<n>While multiple agents can accelerate learning by sharing information, strategic agents might prefer to free-ride and avoid exploration.<n>We propose textttCAOS, a mechanism that sustains collaboration as a Nash equilibrium while achieving strong regret guarantees.
  arXiv  Detail & Related papers  (2026-05-13T08:10:36Z)
• CoopEval: Benchmarking Cooperation-Sustaining Mechanisms and LLM Agents in Social Dilemmas [37.9285901060329]
  LLMs with stronger reasoning capabilities behave _less_ cooperatively in mixed-motive games.<n>Our experiments show that recent models consistently defect in single-shot social dilemmas.<n>Contracting and mediation are most effective in achieving cooperative outcomes between capable LLM models.
  arXiv  Detail & Related papers  (2026-04-16T17:40:30Z)
• Corrupted by Reasoning: Reasoning Language Models Become Free-Riders in Public Goods Games [87.5673042805229]
  How large language models balance self-interest and collective well-being is a critical challenge for ensuring alignment, robustness, and safe deployment.<n>We adapt a public goods game with institutional choice from behavioral economics, allowing us to observe how different LLMs navigate social dilemmas.<n>Surprisingly, we find that reasoning LLMs, such as the o1 series, struggle significantly with cooperation.
  arXiv  Detail & Related papers  (2025-06-29T15:02:47Z)
• Cognitive Insights and Stable Coalition Matching for Fostering Multi-Agent Cooperation [6.536780912510439]
  We propose a novel matching coalition mechanism that leverages the strengths of agents with different ToM levels. Our work demonstrates the potential of leveraging ToM to create more sophisticated and human-like coordination strategies.
  arXiv  Detail & Related papers  (2024-05-28T10:59:33Z)
• Reaching Consensus in Cooperative Multi-Agent Reinforcement Learning with Goal Imagination [16.74629849552254]
  We propose a model-based consensus mechanism to explicitly coordinate multiple agents. The proposed Multi-agent Goal Imagination (MAGI) framework guides agents to reach consensus with an Imagined common goal. We show that such efficient consensus mechanism can guide all agents cooperatively reaching valuable future states.
  arXiv  Detail & Related papers  (2024-03-05T18:07:34Z)
• Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation [52.930183136111864]
  We propose using scorable negotiation to evaluate Large Language Models (LLMs) To reach an agreement, agents must have strong arithmetic, inference, exploration, and planning capabilities. We provide procedures to create new games and increase games' difficulty to have an evolving benchmark.
  arXiv  Detail & Related papers  (2023-09-29T13:33:06Z)
• ProAgent: Building Proactive Cooperative Agents with Large Language Models [89.53040828210945]
  ProAgent is a novel framework that harnesses large language models to create proactive agents. ProAgent can analyze the present state, and infer the intentions of teammates from observations. ProAgent exhibits a high degree of modularity and interpretability, making it easily integrated into various coordination scenarios.
  arXiv  Detail & Related papers  (2023-08-22T10:36:56Z)
• Adaptive Value Decomposition with Greedy Marginal Contribution Computation for Cooperative Multi-Agent Reinforcement Learning [48.41925886860991]
  Real-world cooperation often requires intensive coordination among agents simultaneously. Traditional methods that learn the value function as a monotonic mixing of per-agent utilities cannot solve the tasks with non-monotonic returns. We propose a novel explicit credit assignment method to address the non-monotonic problem.
  arXiv  Detail & Related papers  (2023-02-14T07:23:59Z)
• On the Critical Role of Conventions in Adaptive Human-AI Collaboration [73.21967490610142]
  We propose a learning framework that teases apart rule-dependent representation from convention-dependent representation. We experimentally validate our approach on three collaborative tasks varying in complexity.
  arXiv  Detail & Related papers  (2021-04-07T02:46:19Z)
• Efficient Querying for Cooperative Probabilistic Commitments [29.57444821831916]
  Multiagent systems can use commitments as the core of a general coordination infrastructure. We show how cooperative agents can efficiently find an (approximately) optimal commitment by querying about carefully-selected commitment choices.
  arXiv  Detail & Related papers  (2020-12-14T00:47:09Z)
• Negotiating Team Formation Using Deep Reinforcement Learning [32.066083116314815]
  We propose a framework for training agents to negotiate and form teams using deep reinforcement learning. We evaluate our approach on both non-spatial and spatially extended team-formation negotiation environments.
  arXiv  Detail & Related papers  (2020-10-20T15:41:23Z)
• UneVEn: Universal Value Exploration for Multi-Agent Reinforcement Learning [53.73686229912562]
  We propose a novel MARL approach called Universal Value Exploration (UneVEn) UneVEn learns a set of related tasks simultaneously with a linear decomposition of universal successor features. Empirical results on a set of exploration games, challenging cooperative predator-prey tasks requiring significant coordination among agents, and StarCraft II micromanagement benchmarks show that UneVEn can solve tasks where other state-of-the-art MARL methods fail.
  arXiv  Detail & Related papers  (2020-10-06T19:08:47Z)
• A Cordial Sync: Going Beyond Marginal Policies for Multi-Agent Embodied Tasks [111.34055449929487]
  We introduce the novel task FurnMove in which agents work together to move a piece of furniture through a living room to a goal. Unlike existing tasks, FurnMove requires agents to coordinate at every timestep. We identify two challenges when training agents to complete FurnMove: existing decentralized action sampling procedures do not permit expressive joint action policies. Using SYNC-policies and CORDIAL, our agents achieve a 58% completion rate on FurnMove, an impressive absolute gain of 25 percentage points over competitive decentralized baselines.
  arXiv  Detail & Related papers  (2020-07-09T17:59:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.