Multi-Agent Reinforcement Learning for Market Making: Competition without Collusion

• URL: http://arxiv.org/abs/2510.25929v1
• Date: Wed, 29 Oct 2025 20:07:47 GMT
• Title: Multi-Agent Reinforcement Learning for Market Making: Competition without Collusion
• Abstract summary: We propose a hierarchical multi-agent reinforcement learning framework to study algorithmic collusion in market making.<n>The framework includes a self-interested market maker (AgentA), which is trained in an uncertain environment shaped by an adversary.<n>We show that adaptive incentive control supports more sustainable strategic co-existence in heterogeneous agent environments.
• Score: 6.598173855286935
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Algorithmic collusion has emerged as a central question in AI: Will the interaction between different AI agents deployed in markets lead to collusion? More generally, understanding how emergent behavior, be it a cartel or market dominance from more advanced bots, affects the market overall is an important research question. We propose a hierarchical multi-agent reinforcement learning framework to study algorithmic collusion in market making. The framework includes a self-interested market maker (Agent~A), which is trained in an uncertain environment shaped by an adversary, and three bottom-layer competitors: the self-interested Agent~B1 (whose objective is to maximize its own PnL), the competitive Agent~B2 (whose objective is to minimize the PnL of its opponent), and the hybrid Agent~B$^\star$, which can modulate between the behavior of the other two. To analyze how these agents shape the behavior of each other and affect market outcomes, we propose interaction-level metrics that quantify behavioral asymmetry and system-level dynamics, while providing signals potentially indicative of emergent interaction patterns. Experimental results show that Agent~B2 secures dominant performance in a zero-sum setting against B1, aggressively capturing order flow while tightening average spreads, thus improving market execution efficiency. In contrast, Agent~B$^\star$ exhibits a self-interested inclination when co-existing with other profit-seeking agents, securing dominant market share through adaptive quoting, yet exerting a milder adverse impact on the rewards of Agents~A and B1 compared to B2. These findings suggest that adaptive incentive control supports more sustainable strategic co-existence in heterogeneous agent environments and offers a structured lens for evaluating behavioral design in algorithmic trading systems.

Related papers

• Poisoning Agentic Alpha: Adversarial Vulnerabilities Across Roles and Architectures in Multi-Agent Trading Systems [103.90831279037094]
  LLM-based multi-agent trading systems are moving rapidly from research prototypes to live deployments that control real assets.<n>A corrupted signal can propagate to the final decision and translate into realized financial loss.<n>We present the first systematic empirical study in the financial domain to characterize how an adversarial signal enters a multi-agent trading system.
  arXiv  Detail & Related papers  (2026-08-25T05:07:37Z)
• Adversarial Stress Testing of Role-Playing Language Agents using Multi-Agent Evaluation [0.0]
  Role-Playing Language Agents (RPLAs) are increasingly deployed in high-stakes applications.<n>Existing evaluation approaches rely on static benchmarks or isolated single-turn prompts.<n>We present a modular multi-agent platform for adversarially stress-testing RPLAs through structured, multi-turn dialogue.
  arXiv  Detail & Related papers  (2026-08-04T05:54:32Z)
• Formal Mechanisms for Market Stability in Self-Interested Agent Societies: A Marketplace Simulation Study [3.2232118075036573]
  Self-interested agents, left unconstrained, tend toward defection in repeated social dilemmas, causing cooperative gains from trade to collapse.<n>This paper investigates what formal mechanisms, layered on top of unrestricted communication, are sufficient for a society of such agents to maintain market stability.
  arXiv  Detail & Related papers  (2026-07-09T16:21:16Z)
• Taming Actor-Observer Asymmetry in Agents via Dialectical Alignment [59.536125286960186]
  Multi-agent frameworks assigning specialized roles are increasingly adopted to enable self-reflection and mutual auditing.<n>We find it simultaneously induces a human-like cognitive bias known as Actor-Observer Asymmetry.<n>ReTAS guides agents to synthesize conflicting viewpoints into an objective consensus.
  arXiv  Detail & Related papers  (2026-04-21T15:05:58Z)
• AffectAgent: Collaborative Multi-Agent Reasoning for Retrieval-Augmented Multimodal Emotion Recognition [62.16431420189863]
  LLM-based multimodal emotion recognition relies on static parametric memory and often hallucinates when interpreting nuanced affective states.<n>We introduce AffectAgent, an affect-oriented multi-agent retrieval-augmented generation framework.<n>AffectAgent comprises three jointly optimized agents, namely a query planner, an evidence filter, and an emotion generator.
  arXiv  Detail & Related papers  (2026-04-14T13:49:19Z)
• Magentic Marketplace: An Open-Source Environment for Studying Agentic Markets [74.91125572848439]
  We study two-sided agentic marketplaces where Assistant agents represent consumers and Service agents represent competing businesses.<n>This environment enables us to study key market dynamics: the utility agents achieve, behavioral biases, vulnerability to manipulation, and how search mechanisms shape market outcomes.<n>Our experiments show that frontier models can approach optimal welfare-- but only under ideal search conditions. Performance degrades sharply with scale, and all models exhibit severe first-proposal bias, creating 10-30x advantages for response speed over quality.
  arXiv  Detail & Related papers  (2025-10-27T18:35:59Z)
• AdvEvo-MARL: Shaping Internalized Safety through Adversarial Co-Evolution in Multi-Agent Reinforcement Learning [78.5751183537704]
  AdvEvo-MARL is a co-evolutionary multi-agent reinforcement learning framework that internalizes safety into task agents.<n>Rather than relying on external guards, AdvEvo-MARL jointly optimize attackers and defenders.
  arXiv  Detail & Related papers  (2025-10-02T02:06:30Z)
• Can an Individual Manipulate the Collective Decisions of Multi-Agents? [53.01767232004823]
  M-Spoiler is a framework that simulates agent interactions within a multi-agent system to generate adversarial samples.<n>M-Spoiler introduces a stubborn agent that actively aids in optimizing adversarial samples.<n>Our findings confirm the risks posed by the knowledge of an individual agent in multi-agent systems.
  arXiv  Detail & Related papers  (2025-09-20T01:54:20Z)
• Vulnerable Agent Identification in Large-Scale Multi-Agent Reinforcement Learning [49.31650627835956]
  Partial agent failure becomes inevitable when systems scale up, making it crucial to identify the subset of agents whose compromise would most severely degrade overall performance.<n>In this paper, we study this Vulnerable Agent Identification (VAI) problem in large-scale multi-agent reinforcement learning (MARL)<n> Experiments show our method effectively identifies more vulnerable agents in large-scale MARL and the rule-based system, fooling system into worse failures, and learning a value function that reveals the vulnerability of each agent.
  arXiv  Detail & Related papers  (2025-09-18T16:03:50Z)
• Evaluating LLM Agent Collusion in Double Auctions [1.3194391758295114]
  We study the behavior of large language models (LLMs) acting as sellers in simulated double auction markets.<n>We find that direct seller communication increases collusive tendencies, the propensity to collude varies across models, and environmental pressures, such as oversight and urgency from authority figures, influence collusive behavior.
  arXiv  Detail & Related papers  (2025-07-02T07:06:49Z)
• DCIR: Dynamic Consistency Intrinsic Reward for Multi-Agent Reinforcement Learning [84.22561239481901]
  We propose a new approach that enables agents to learn whether their behaviors should be consistent with that of other agents. We evaluate DCIR in multiple environments including Multi-agent Particle, Google Research Football and StarCraft II Micromanagement.
  arXiv  Detail & Related papers  (2023-12-10T06:03:57Z)
• ProAgent: Building Proactive Cooperative Agents with Large Language Models [89.53040828210945]
  ProAgent is a novel framework that harnesses large language models to create proactive agents. ProAgent can analyze the present state, and infer the intentions of teammates from observations. ProAgent exhibits a high degree of modularity and interpretability, making it easily integrated into various coordination scenarios.
  arXiv  Detail & Related papers  (2023-08-22T10:36:56Z)
• Robustness Testing for Multi-Agent Reinforcement Learning: State Perturbations on Critical Agents [2.5204420653245245]
  Multi-Agent Reinforcement Learning (MARL) has been widely applied in many fields such as smart traffic and unmanned aerial vehicles. This work proposes a novel Robustness Testing framework for MARL that attacks states of Critical Agents.
  arXiv  Detail & Related papers  (2023-06-09T02:26:28Z)
• Artificial Intelligence and Dual Contract [2.1756081703276]
  We develop a model where two principals, each equipped with independent Q-learning algorithms, interact with a single agent. Our findings reveal that the strategic behavior of AI principals hinges crucially on the alignment of their profits.
  arXiv  Detail & Related papers  (2023-03-22T07:31:44Z)
• Multi-Issue Bargaining With Deep Reinforcement Learning [0.0]
  This paper evaluates the use of deep reinforcement learning in bargaining games. Two actor-critic networks were trained for the bidding and acceptance strategy. Neural agents learn to exploit time-based agents, achieving clear transitions in decision preference values. They also demonstrate adaptive behavior against different combinations of concession, discount factors, and behavior-based strategies.
  arXiv  Detail & Related papers  (2020-02-18T18:33:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.