Measuring Fine-Grained Negotiation Tactics of Humans and LLMs in Diplomacy

• URL: http://arxiv.org/abs/2512.18292v1
• Date: Sat, 20 Dec 2025 09:33:55 GMT
• Title: Measuring Fine-Grained Negotiation Tactics of Humans and LLMs in Diplomacy
• Authors: Wenkai Li, Lynnette Hui Xian Ng, Andy Liu, Daniel Fried,
• Abstract summary: The study of negotiation styles dates back to Aristotle's ethos-pathos-logos rhetoric.<n>Our focus is the strategic dialogue board game Diplomacy, which affords rich natural language negotiation.
• Score: 30.530636602009263
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The study of negotiation styles dates back to Aristotle's ethos-pathos-logos rhetoric. Prior efforts primarily studied the success of negotiation agents. Here, we shift the focus towards the styles of negotiation strategies. Our focus is the strategic dialogue board game Diplomacy, which affords rich natural language negotiation and measures of game success. We used LLM-as-a-judge to annotate a large human-human set of Diplomacy games for fine-grained negotiation tactics from a sociologically-grounded taxonomy. Using a combination of the It Takes Two and WebDiplomacy datasets, we demonstrate the reliability of our LLM-as-a-Judge framework and show strong correlations between negotiation features and success in the Diplomacy setting. Lastly, we investigate the differences between LLM and human negotiation strategies and show that fine-tuning can steer LLM agents toward more human-like negotiation behaviors.

Related papers

• MERIT Feedback Elicits Better Bargaining in LLM Negotiators [38.1466669265123]
  AgoraBench is a new benchmark spanning nine challenging settings.<n>This is operationalized via agent utility, negotiation power, and acquisition ratio that implicitly measure how well the negotiation aligns with human preference.<n>Our mechanism substantially improves negotiation performance, yielding deeper strategic behavior and stronger opponent awareness.
  arXiv  Detail & Related papers  (2026-02-11T03:09:45Z)
• GameTalk: Training LLMs for Strategic Conversation [51.29670609281524]
  We introduce textbfGameTalk, a framework for training LLMs to make strategic decisions via multi-turn interactions.<n>Unlike prior work that focuses on single-turn objectives or static action prediction, we train LLMs to optimize a global objective across full conversations.<n>We evaluate this approach on a suite of increasingly complex games, designed to stress different aspects of reasoning, coordination, and opponent modeling.
  arXiv  Detail & Related papers  (2026-01-22T19:18:39Z)
• LLM Rationalis? Measuring Bargaining Capabilities of AI Negotiators [2.1952520391635586]
  Bilateral negotiation is a complex, context-sensitive task in which human negotiators dynamically adjust anchors, pacing, and flexibility to exploit power asymmetries and informal cues.<n>We introduce a unified mathematical framework for modeling concession dynamics based on a hyperbolic tangent curve.<n>We conduct a large-scale empirical comparison between human negotiators and four state-of-the-art large language models (LLMs) across natural-language and numeric-offers settings.
  arXiv  Detail & Related papers  (2025-12-15T07:50:09Z)
• EvoEmo: Towards Evolved Emotional Policies for Adversarial LLM Agents in Multi-Turn Price Negotiation [61.627248012799704]
  Existing Large Language Models (LLMs) agents largely overlook the functional role of emotions in such negotiations.<n>We present EvoEmo, an evolutionary reinforcement learning framework that optimize dynamic emotional expression in negotiations.
  arXiv  Detail & Related papers  (2025-09-04T15:23:58Z)
• Are LLMs Effective Negotiators? Systematic Evaluation of the Multifaceted Capabilities of LLMs in Negotiation Dialogues [4.738985706520995]
  This work aims to systematically analyze the multifaceted capabilities of LLMs across diverse dialogue scenarios. Our analysis highlights GPT-4's superior performance in many tasks while identifying specific challenges.
  arXiv  Detail & Related papers  (2024-02-21T06:11:03Z)
• How Well Can LLMs Negotiate? NegotiationArena Platform and Analysis [50.15061156253347]
  Negotiation is the basis of social interactions; humans negotiate everything from the price of cars to how to share common resources. With rapidly growing interest in using large language models (LLMs) to act as agents on behalf of human users, such LLM agents would also need to be able to negotiate. We develop NegotiationArena: a flexible framework for evaluating and probing the negotiation abilities of LLM agents.
  arXiv  Detail & Related papers  (2024-02-08T17:51:48Z)
• Assistive Large Language Model Agents for Socially-Aware Negotiation Dialogues [47.977032883078664]
  We develop assistive agents based on Large Language Models (LLMs) that aid interlocutors in business negotiations.<n>A third LLM acts as a remediator agent to rewrite utterances violating norms for improving negotiation outcomes.<n>We provide rich empirical evidence to demonstrate its effectiveness in negotiations across three different negotiation topics.
  arXiv  Detail & Related papers  (2024-01-29T09:07:40Z)
• Negotiating with LLMS: Prompt Hacks, Skill Gaps, and Reasoning Deficits [1.2818275315985972]
  We conduct a user study engaging over 40 individuals across all age groups in price negotiations with an LLM. We show that the negotiated prices humans manage to achieve span a broad range, which points to a literacy gap in effectively interacting with LLMs.
  arXiv  Detail & Related papers  (2023-11-26T08:44:58Z)
• Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation [52.930183136111864]
  We propose using scorable negotiation to evaluate Large Language Models (LLMs) To reach an agreement, agents must have strong arithmetic, inference, exploration, and planning capabilities. We provide procedures to create new games and increase games' difficulty to have an evolving benchmark.
  arXiv  Detail & Related papers  (2023-09-29T13:33:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.