Emergent Misaligned Communication in Long-Horizon Multi-Agent LLM Commerce

• URL: http://arxiv.org/abs/2608.14825v2
• Date: Tue, 18 Aug 2026 01:55:42 GMT
• Title: Emergent Misaligned Communication in Long-Horizon Multi-Agent LLM Commerce
• Abstract summary: We operationalize speech-act misalignment as emails containing false factual claims, manipulation, collusion, or threats.<n>12.6% of emails are labeled misaligned; misalignment appears in all 20 runs and 74.7% of individual agent-runs.<n>We find no evidence that higher-capability models differentially exploit weaker counterparties.
• Score: 39.937871162878785
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Frontier LLM agents increasingly transact on behalf of separate principals, often using natural language rather than structured APIs. Much of the safety literature studies misaligned LLM behavior through adversarial-elicitation evaluations on single agents or stylized tasks. Its prevalence and structure in settings that combine long horizons, separate principals, real operational state, and inter-agent natural-language exchange remain insufficiently measured. We study 2,583 inter-agent emails from 20 one-year simulation runs of Vending-Bench Arena, a competitive vending environment spanning 13 frontier LLMs. We operationalize speech-act misalignment as emails containing false factual claims, manipulation, collusion, or threats, combining message content with ground-truth simulator state and logged reasoning traces to classify and validate such behavior. Under our primary classifier, 12.6% of emails are labeled misaligned; misalignment appears in all 20 runs and 74.7% of individual agent-runs. Both the magnitude and composition of this misalignment are preserved under repeated classification at different sampling temperatures and under full-pipeline replication with judges from two other frontier-model families. Misalignment is also reciprocal and stress-conditioned: receiving a misaligned email from a counterparty raises the odds of a misaligned reply by 1.65x, and low-inventory conditions raise them by 1.58x. Across tests of capability-asymmetric exploitation, we find no evidence that higher-capability models differentially exploit weaker counterparties, and model performance rank does not predict misalignment rates. Together, these results indicate that measurable, state-dependent misalignment can arise in competitive multi-agent environments without engineered elicitation, in patterns associated with operational scarcity and counterparty behavior rather than model capability alone.

Related papers

• Confidently Wrong: Exception Chain Collapse in Frontier LLM Rule Evaluation [0.0]
  We document a failure class in frontier large language models.<n>The failure reproduces at first observation, but its empirical surface is unstable.<n>For regulated, frontier-model accuracy is a moving compliance boundary that shifts without notice.
  arXiv  Detail & Related papers  (2026-07-25T22:40:02Z)
• Probing the Misaligned Thinking Process of Language Models [32.29324298373053]
  Large language models exhibit a growing range of misaligned behaviors.<n>It is critical to reliably detect such behaviors to ensure safe and responsible use.<n>We propose to monitor misalignment by decomposing it into fine-grained cognitive processes.
  arXiv  Detail & Related papers  (2026-06-23T07:40:28Z)
• INTACT: Ego-Guided Typed Sparse Evidence Retrieval for Heterogeneous Collaborative Perception [50.64511548643398]
  We propose an ego-guided typed sparse evidence retrieval framework for heterogeneous perception.<n>IntACT lets the ego vehicle issue typed evidence queries that express suspected objects and evidence-deficient regions.<n>Experiments on simulated and real-world heterogeneous perception benchmarks validate the effectiveness and deployability of INTACT.
  arXiv  Detail & Related papers  (2026-06-03T04:34:53Z)
• ClawForge: Generating Executable Interactive Benchmarks for Command-Line Agents [59.626170560327274]
  We present textbfClawForge, a generator-backed benchmark framework for executable command-line categories under state conflict.<n>We instantiate this framework as the ClawForge-Bench (17 scenarios, 6 ability categories)
  arXiv  Detail & Related papers  (2026-05-13T21:34:08Z)
• Taming Actor-Observer Asymmetry in Agents via Dialectical Alignment [59.536125286960186]
  Multi-agent frameworks assigning specialized roles are increasingly adopted to enable self-reflection and mutual auditing.<n>We find it simultaneously induces a human-like cognitive bias known as Actor-Observer Asymmetry.<n>ReTAS guides agents to synthesize conflicting viewpoints into an objective consensus.
  arXiv  Detail & Related papers  (2026-04-21T15:05:58Z)
• OccuBench: Evaluating AI Agents on Real-World Professional Tasks via Language Environment Simulation [57.505743202759646]
  OccuBench is a benchmark covering 100 real-world professional task scenarios across 10 industry categories and 65 specialized domains.<n>Our multi-agent synthesis pipeline automatically produces evaluation instances with guaranteed solvability, calibrated difficulty, and document-grounded diversity.
  arXiv  Detail & Related papers  (2026-04-13T00:27:32Z)
• Can AI Agents Agree? [32.75269650141292]
  Large language models are increasingly deployed as cooperating agents, yet their behavior in adversarial consensus settings has not been studied.<n>We evaluate LLM-based agents on a Byzantine consensus game over scalar values using a synchronous all-to-all simulation.<n>We find that valid agreement is not reliable even in benign settings and degrades as group size grows.
  arXiv  Detail & Related papers  (2026-03-01T18:18:59Z)
• Are Aligned Large Language Models Still Misaligned? [13.062124372682106]
  Mis-Align Bench is a unified benchmark for analyzing misalignment across safety, value, and cultural dimensions.<n> SAVACU is an English-aligned dataset of 382,424 misaligned samples spanning 112 domains (or labels)
  arXiv  Detail & Related papers  (2026-02-11T19:30:43Z)
• Alignment Tipping Process: How Self-Evolution Pushes LLM Agents Off the Rails [103.05296856071931]
  We identify the Alignment Tipping Process (ATP), a critical post-deployment risk unique to self-evolving Large Language Model (LLM) agents.<n>ATP arises when continual interaction drives agents to abandon alignment constraints established during training in favor of reinforced, self-interested strategies.<n>Our experiments show that alignment benefits erode rapidly under self-evolution, with initially aligned models converging toward unaligned states.
  arXiv  Detail & Related papers  (2025-10-06T14:48:39Z)
• AgentMisalignment: Measuring the Propensity for Misaligned Behaviour in LLM-Based Agents [0.0]
  Large Language Model (LLM) agents become more widespread, associated misalignment risks increase.<n>In this work, we approach misalignment as a conflict between the internal goals pursued by the model and the goals intended by its deployer.<n>We introduce a misalignment propensity benchmark, textscAgentMisalignment, a benchmark suite designed to evaluate the propensity of LLM agents to misalign in realistic scenarios.
  arXiv  Detail & Related papers  (2025-06-04T14:46:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.