Used Car Salesbots? Honesty and Credulity of LLMs as Bargaining Agents under Partial Information

• URL: http://arxiv.org/abs/2605.31445v1
• Date: Fri, 29 May 2026 15:40:29 GMT
• Title: Used Car Salesbots? Honesty and Credulity of LLMs as Bargaining Agents under Partial Information
• Abstract summary: We study agents in simulated bargaining scenarios, where a buyer and a seller communicate through a text channel.<n>We evaluate their performance and investigate their honesty as well as their credulity.<n>We find that off-the-shelf LLMs all substantially deviate from game-theoretical equilibria.<n>Fine-tuning on financial utility makes the agents stronger at achieving better deals but also more dishonest.
• Score: 26.60109719186885
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this work we study agents in simulated bargaining scenarios, where a buyer and a seller communicate through a text channel and attempt to negotiate mutually beneficial trades, under different information regimes (complete information, information asymmetry or mutual uncertainty). We evaluate their performance w.r.t. game-theoretical solutions and further investigate their honesty (their tendency to disclose or withhold information or to mislead and deceive) as well as their credulity (their tendency to trust or distrust information provided by the other agent). We study zero-shot LLM agents with simple prompting scaffolding as well as fine-tuned agents, in order to investigate whether optimising the agents to maximise financial profits makes them stronger negotiators but also more dishonest and less trusting. We find that off-the-shelf LLMs all substantially deviate from game-theoretical equilibria, they attempt to lie about their private information but cannot efficiently exploit information asymmetries. Fine-tuning on financial utility makes the agents stronger at achieving better deals but also more dishonest, highlighting the risks that optimising agents for a task can have on their safety. We release our code and a dataset of bargaining scenarios.

Related papers

• PrivacyPeek: Auditing What LLM-Based Agents Acquire, Not Just What They Say [48.84133320567554]
  LLM-based agents are rapidly advancing, autonomously invoking external tools to complete tasks for users.<n> Existing privacy benchmarks audit what the agent's response or outgoing actions disclose, but overlook the acquisition stage where data first enters the agent's context.<n>We introduce emphPrivacyPeek, a benchmark for evaluating acquisition-stage privacy leakage of LLM-based agents.
  arXiv  Detail & Related papers  (2026-05-29T04:55:12Z)
• Counterparty Modeling is Not Strategy: The Limits of LLM Negotiators [29.84830992033132]
  We study whether large language model (LLM) agents do this in a controlled multi-attribute bargaining environment.<n>We find that current LLM agents can model a counterparty's preferences, but do not reliably turn that knowledge into strategic bargaining.<n>Because agents fail to leverage this underlying utility structure for strategic advantage, their final agreements are heavily dictated by surface-level opening anchors rather than actual utility weights.
  arXiv  Detail & Related papers  (2026-05-15T19:27:05Z)
• Instructing LLMs to Negotiate using Reinforcement Learning with Verifiable Rewards [45.56436052535799]
  We investigate if Reinforcement Learning from Verifiable Rewards can effectively teach Large Language Models to negotiate.<n>We introduce a framework that trains a mid-sized buyer agent against a regulated seller across a wide distribution of real-world products.<n>Our results demonstrate that this verifiable training allows a 30B agent to significantly outperform frontier models over ten times its size in extracting surplus.
  arXiv  Detail & Related papers  (2026-04-10T19:35:39Z)
• Are Your Agents Upward Deceivers? [73.1073084327614]
  Large Language Model (LLM)-based agents are increasingly used as autonomous subordinates that carry out tasks for users.<n>This raises the question of whether they may also engage in deception, similar to how individuals in human organizations lie to superiors to create a good image or avoid punishment.<n>We observe and define agentic upward deception, a phenomenon in which an agent facing environmental constraints conceals its failure and performs actions that were not requested without reporting.
  arXiv  Detail & Related papers  (2025-12-04T14:47:05Z)
• Simulating Misinformation Vulnerabilities With Agent Personas [1.0120858915885353]
  We develop an agent-based simulation using Large Language Models to model responses to misinformation.<n>We construct agent personas spanning five professions and three mental schemas, and evaluate their reactions to news headlines.<n>Our findings show that LLM-generated agents align closely with ground-truth labels and human predictions, supporting their use as proxies for studying information responses.
  arXiv  Detail & Related papers  (2025-10-31T18:44:00Z)
• Balancing Truthfulness and Informativeness with Uncertainty-Aware Instruction Fine-Tuning [79.48839334040197]
  Instruction fine-tuning (IFT) can increase the informativeness of large language models (LLMs), but may reduce their truthfulness.<n>In this paper, we empirically demonstrate how unfamiliar knowledge in IFT datasets can negatively affect the truthfulness of LLMs.<n>We introduce two new IFT paradigms, $UNIT_cut$ and $UNIT_ref$, to address this issue.
  arXiv  Detail & Related papers  (2025-02-17T16:10:30Z)
• AI-LieDar: Examine the Trade-off Between Utility and Truthfulness in LLM Agents [27.10147264744531]
  We study how Large Language Models (LLM)-based agents navigate scenarios in a multi-turn interactive setting.<n>We develop a truthfulness detector inspired by psychological literature to assess the agents' responses.<n>Our experiment demonstrates that all models are truthful less than 50% of the time, though truthfulness and goal achievement (utility) rates vary across models.
  arXiv  Detail & Related papers  (2024-09-13T17:41:12Z)
• How Well Can LLMs Negotiate? NegotiationArena Platform and Analysis [50.15061156253347]
  Negotiation is the basis of social interactions; humans negotiate everything from the price of cars to how to share common resources. With rapidly growing interest in using large language models (LLMs) to act as agents on behalf of human users, such LLM agents would also need to be able to negotiate. We develop NegotiationArena: a flexible framework for evaluating and probing the negotiation abilities of LLM agents.
  arXiv  Detail & Related papers  (2024-02-08T17:51:48Z)
• Can Large Language Model Agents Simulate Human Trust Behavior? [81.45930976132203]
  We investigate whether Large Language Model (LLM) agents can simulate human trust behavior. GPT-4 agents manifest high behavioral alignment with humans in terms of trust behavior. We also probe the biases of agent trust and differences in agent trust towards other LLM agents and humans.
  arXiv  Detail & Related papers  (2024-02-07T03:37:19Z)
• TrustLLM: Trustworthiness in Large Language Models [446.5640421311468]
  This paper introduces TrustLLM, a comprehensive study of trustworthiness in large language models (LLMs) We first propose a set of principles for trustworthy LLMs that span eight different dimensions. Based on these principles, we establish a benchmark across six dimensions including truthfulness, safety, fairness, robustness, privacy, and machine ethics.
  arXiv  Detail & Related papers  (2024-01-10T22:07:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.