Measuring Bargaining Abilities of LLMs: A Benchmark and A Buyer-Enhancement Method

• URL: http://arxiv.org/abs/2402.15813v3
• Date: Tue, 4 Jun 2024 08:12:04 GMT
• Title: Measuring Bargaining Abilities of LLMs: A Benchmark and A Buyer-Enhancement Method
• Authors: Tian Xia, Zhiwei He, Tong Ren, Yibo Miao, Zhuosheng Zhang, Yang Yang, Rui Wang,
• Abstract summary: This paper describes the Bargaining task as an asymmetric incomplete information game. It allows us to quantitatively assess an agent's performance in the Bargain task. We propose a novel approach called OG-Narrator that integrates a deterministic Offer Generator to control the price range of Buyer's offers.
• Score: 17.388837360641276
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Bargaining is an important and unique part of negotiation between humans. As LLM-driven agents learn to negotiate and act like real humans, how to evaluate agents' bargaining abilities remains an open problem. For the first time, we formally described the Bargaining task as an asymmetric incomplete information game, defining the gains of the Buyer and Seller in multiple bargaining processes. It allows us to quantitatively assess an agent's performance in the Bargain task. We collected a real product price dataset, AmazonHistoryPrice, and conducted evaluations of various LLM agents' bargaining abilities. We find that playing a Buyer is much harder than a Seller, and increasing model size can not effectively improve the Buyer's performance. To address the challenge, we propose a novel approach called OG-Narrator that integrates a deterministic Offer Generator to control the price range of Buyer's offers, and an LLM Narrator to create natural language sentences for generated offers. Experimental results show that OG-Narrator improves the buyer's deal rates from 26.67% to 88.88% and brings a ten times multiplication of profits on all baselines, even a model that has not been aligned.

Related papers

• Algorithmic Collusion by Large Language Models [0.08192907805418582]
  We conduct experiments with algorithmic pricing agents based on Large Language Models (LLMs) and GPT-4. We find that LLM-based agents are adept at pricing tasks, (2) LLM-based pricing agents autonomously collude in oligopoly settings to the detriment of consumers, and (3) variation in seemingly innocuous phrases in LLM instructions may increase collusion. Our findings underscore the need for antitrust regulation regarding algorithmic pricing, and uncover regulatory challenges unique to LLM-based pricing agents.
  arXiv  Detail & Related papers  (2024-03-31T21:43:05Z)
• How Well Can LLMs Negotiate? NegotiationArena Platform and Analysis [50.15061156253347]
  Negotiation is the basis of social interactions; humans negotiate everything from the price of cars to how to share common resources. With rapidly growing interest in using large language models (LLMs) to act as agents on behalf of human users, such LLM agents would also need to be able to negotiate. We develop NegotiationArena: a flexible framework for evaluating and probing the negotiation abilities of LLM agents.
  arXiv  Detail & Related papers  (2024-02-08T17:51:48Z)
• Assistive Large Language Model Agents for Socially-Aware Negotiation Dialogues [47.977032883078664]
  We develop assistive agents based on Large Language Models (LLMs) We simulate business negotiations by letting two LLM-based agents engage in role play. A third LLM acts as a remediator agent to rewrite utterances violating norms for improving negotiation outcomes.
  arXiv  Detail & Related papers  (2024-01-29T09:07:40Z)
• Salespeople vs SalesBot: Exploring the Role of Educational Value in Conversational Recommender Systems [78.84530426424838]
  Existing conversational recommender systems often overlook users' lack of background knowledge, focusing solely on gathering preferences. We introduce SalesOps, a framework that facilitates the simulation and evaluation of such systems. We build SalesBot and ShopperBot, a pair of LLM-powered agents that can simulate either side of the framework.
  arXiv  Detail & Related papers  (2023-10-26T19:44:06Z)
• ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate [57.71597869337909]
  We build a multi-agent referee team called ChatEval to autonomously discuss and evaluate the quality of generated responses from different models. Our analysis shows that ChatEval transcends mere textual scoring, offering a human-mimicking evaluation process for reliable assessments.
  arXiv  Detail & Related papers  (2023-08-14T15:13:04Z)
• Learning Multi-Agent Intention-Aware Communication for Optimal Multi-Order Execution in Finance [96.73189436721465]
  We first present a multi-agent RL (MARL) method for multi-order execution considering practical constraints. We propose a learnable multi-round communication protocol, for the agents communicating the intended actions with each other. Experiments on the data from two real-world markets have illustrated superior performance with significantly better collaboration effectiveness.
  arXiv  Detail & Related papers  (2023-07-06T16:45:40Z)
• Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedback [97.54519989641388]
  We study whether multiple large language models (LLMs) can autonomously improve each other in a negotiation game by playing, reflecting, and criticizing. Only a subset of the language models we consider can self-play and improve the deal price from AI feedback.
  arXiv  Detail & Related papers  (2023-05-17T11:55:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.