TERMS-Bench: Diagnosing LLM Negotiation Agents Beyond Deal Rate

• URL: http://arxiv.org/abs/2605.13909v1
• Date: Wed, 13 May 2026 06:22:50 GMT
• Title: TERMS-Bench: Diagnosing LLM Negotiation Agents Beyond Deal Rate
• Abstract summary: Negotiation is a canonical testbed for agentic language models.<n>Unlike math or code, it has no intrinsic verifier.<n>We introduce Terms-Bench, a Bayesian-game framework that makes the environment itself the verifier.
• Score: 34.67393151003599
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Negotiation is a central mechanism of economic exchange, shaping markets, procurement, labor agreements, and resource allocation. It is also a canonical testbed for agentic language models, requiring multi-turn interaction under hidden preferences, strategic communication, and binding constraints. These properties make negotiation hard to evaluate: unlike math or code, it has no intrinsic verifier. Existing LLM negotiation evaluations rely on LLM-vs.-LLM interaction or aggregate outcomes such as deal rate, leaving failures opaque. We introduce Terms-Bench, short for Testbed for Economic Reasoning in Multi-turn Strategy, a Bayesian-game framework that makes the environment itself the verifier by specifying the counterpart's latent type, policy, and payoff structure. We instantiate it in bilateral price negotiation, where the counterpart's private state and simulator policy are hidden from the agent but observable to the evaluator. This turns the counterpart from a black-box opponent into a diagnostic instrument, enabling agent-attributable failure analysis and oracle-reference optimality gaps. Evaluating 13 LLM agents spanning frontier systems from major providers, Terms-Bench turns negotiation evaluation from aggregate ranking into actionable diagnosis: where agents fail, why they fail, and what to strengthen. Empirically, frontier models saturate deal rate yet diverge in surplus extraction, cue use, belief calibration, and compliance, revealing agent-specific bargaining bottlenecks masked by prior benchmarks.

Related papers

• Loom: Weaving Diagnostic Strands into Free-Text Consensus via Embedding-Space Reweighting [0.0]
  We present Loom, a generative consensus framework deployed for real-world Root Cause Analysis (RCA)<n>We show how Loom aggregates open-form hypotheses emitted by modular$s by projecting them into a continuous embedding space, and resolves conflicting signals with an iterative centroid-based reweighting algorithm.<n>We discuss our deployment experience, highlighting lessons learned regarding the trade-offs between agentic depth and inference latency, negative results in redundancy detection, and how deterministic consensus fosters trust among Subject Matter Experts.
  arXiv  Detail & Related papers  (2026-09-02T14:24:32Z)
• CLQT: A Closed-Loop, Cost-Aware, Strategy-Consistent Benchmark for Diagnostic Evaluation of LLM Portfolio-Management Agents [0.9924185669370708]
  We introduce CLQT, which reframes closed-loop trading evaluation as diagnosis rather than ranking.<n> CLQT is a fully closed-loop, cost-aware, strategy-consistent, temporally-gated environment.
  arXiv  Detail & Related papers  (2026-06-29T04:26:30Z)
• SidConArena: An Environment Evaluating Agents in Open-Ended,Positive-Sum Bargaining Game [21.27775046269771]
  We introduce SidConArena, a new benchmark framework for evaluating LLM agents in open-ended, positive-sum bargaining.<n>SidConArena formalizes a multi-player economy as a finite-horizon partially observable game with three coupled phases.<n>We show that stronger frontier models achieve higher economic outcomes, yet agents still misvalue resources, bargain passively, and remain limited in long-horizon investment planning.
  arXiv  Detail & Related papers  (2026-06-24T10:35:31Z)
• Cattle Trade: A Multi-Agent Benchmark for LLM Bluffing, Bidding, and Bargaining [0.5691230599672109]
  We introduce textscCattle Trade, a benchmark for evaluating large language models (LLMs) as agents in strategic reasoning.<n>The benchmark combines auctions, hidden-offer trade challenges (TCs), bargaining, bluffing, opponent modeling, and resource allocation within a single long-horizon game.<n>We evaluate seven cost-efficient language models and three deterministic code agents across 242 games.
  arXiv  Detail & Related papers  (2026-05-14T08:20:03Z)
• ComplexMCP: Evaluation of LLM Agents in Dynamic, Interdependent, and Large-Scale Tool Sandbox [61.862814740220806]
  $textbfComplexMCP$ is a benchmark designed to evaluate agents in rigorous conditions.<n>Built on the Model Context Protocol (MCP), $textbfComplexMCP$ provides over 300 meticulously tested tools.
  arXiv  Detail & Related papers  (2026-05-11T16:20:51Z)
• Anchor-and-Resume Concession Under Dynamic Pricing for LLM-Augmented Freight Negotiation [5.225967186005105]
  Deriving $$ from the live spread enables adaptation but introduces a new problem: a pricing shift can cause the formula to retract a previous offer, violating monotonicity.<n>We propose a two-index anchor-and-resume framework that addresses both limitations.<n>A spread-derived $$ maps each load's margin structure to the correct concession posture, while the anchor-and-resume mechanism guarantees monotonically nondecreasing offers under arbitrary pricing shifts.
  arXiv  Detail & Related papers  (2026-04-22T16:17:07Z)
• Training Language Models for Bilateral Trade with Private Information [47.25254982913256]
  Bilateral trade demands individual, strategic surplus rationality, and cooperation to realize gains from trade.<n>We develop a structured bargaining environment where large language models negotiate via tool calls within an event-driven simulator.<n>The environment serves two purposes: a benchmark for frontier models and as a training environment for open-weight models via reinforcement learning.
  arXiv  Detail & Related papers  (2026-04-10T03:04:20Z)
• Conv-FinRe: A Conversational and Longitudinal Benchmark for Utility-Grounded Financial Recommendation [33.09732020656996]
  We introduce Conv-FinRe, a benchmark for stock recommendation that evaluates LLMs beyond behavior matching.<n>We build the benchmark from real market data and human decision trajectories, instantiate controlled advisory conversations, and evaluate a suite of state-of-the-art LLMs.<n>Results reveal a persistent tension between rational decision quality and behavioral alignment.
  arXiv  Detail & Related papers  (2026-02-19T01:29:50Z)
• AgenticPay: A Multi-Agent LLM Negotiation System for Buyer-Seller Transactions [49.49718899185783]
  Large language model (LLM)-based agents are increasingly expected to negotiate, coordinate, and transact autonomously.<n>We introduce AgenticPay, a benchmark and simulation framework for multi-agent buyer-seller negotiation driven by natural language.
  arXiv  Detail & Related papers  (2026-02-05T18:50:36Z)
• PieArena: Frontier Language Agents Achieve MBA-Level Negotiation Performance and Reveal Novel Behavioral Differences [13.759960839511807]
  We introduce PieArena, a large-scale negotiation benchmark grounded in multi-agent interactions.<n>We find systematic evidence of human-expert-level performance in which a representative frontier language agent (GPT-5) matches or outperforms trained business-school students.
  arXiv  Detail & Related papers  (2026-02-05T04:52:20Z)
• From Sycophancy to Sensemaking: Premise Governance for Human-AI Decision Making [0.0]
  Low-friction assistants can become sycophantic, baking in implicit assumptions and pushing verification costs onto experts.<n>We argue reliable human-AI partnership requires a shift from answer generation to collaborative premise governance.
  arXiv  Detail & Related papers  (2026-02-02T17:42:54Z)
• Are You Getting What You Pay For? Auditing Model Substitution in LLM APIs [71.7892165868749]
  Commercial Large Language Model (LLM) APIs create a fundamental trust problem.<n>Users pay for specific models but have no guarantee that providers deliver them faithfully.<n>We formalize this model substitution problem and evaluate detection methods under realistic adversarial conditions.<n>We propose and evaluate the use of Trusted Execution Environments (TEEs) as one practical and robust solution.
  arXiv  Detail & Related papers  (2025-04-07T03:57:41Z)
• EmoDebt: Bayesian-Optimized Emotional Intelligence for Strategic Agent-to-Agent Debt Recovery [65.30120701878582]
  Large Language Model (LLM) agents are vulnerable to exploitation in emotion-sensitive domains like debt collection.<n>EmoDebt is an emotional intelligence engine that reframes a model's ability to express emotion in negotiation as a sequential decision-making problem.<n>EmoDebt achieves significant strategic robustness, substantially outperforming non-adaptive and emotion-agnostic baselines.
  arXiv  Detail & Related papers  (2025-03-27T01:41:34Z)
• Assistive Large Language Model Agents for Socially-Aware Negotiation Dialogues [47.977032883078664]
  We develop assistive agents based on Large Language Models (LLMs) that aid interlocutors in business negotiations.<n>A third LLM acts as a remediator agent to rewrite utterances violating norms for improving negotiation outcomes.<n>We provide rich empirical evidence to demonstrate its effectiveness in negotiations across three different negotiation topics.
  arXiv  Detail & Related papers  (2024-01-29T09:07:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.