Deep Q-Learning Market Makers in a Multi-Agent Simulated Stock Market

• URL: http://arxiv.org/abs/2112.04494v1
• Date: Wed, 8 Dec 2021 14:55:21 GMT
• Title: Deep Q-Learning Market Makers in a Multi-Agent Simulated Stock Market
• Authors: Oscar Fern\'andez Vicente, Fernando Fern\'andez Rebollo, Francisco Javier Garc\'ia Polo
• Abstract summary: This paper focuses precisely on the study of these markets makers strategies from an agent-based perspective. We propose the application of Reinforcement Learning (RL) for the creation of intelligent market markers in simulated stock markets.
• Score: 58.720142291102135
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Market makers play a key role in financial markets by providing liquidity. They usually fill order books with buy and sell limit orders in order to provide traders alternative price levels to operate. This paper focuses precisely on the study of these markets makers strategies from an agent-based perspective. In particular, we propose the application of Reinforcement Learning (RL) for the creation of intelligent market markers in simulated stock markets. This research analyzes how RL market maker agents behaves in non-competitive (only one RL market maker learning at the same time) and competitive scenarios (multiple RL market markers learning at the same time), and how they adapt their strategies in a Sim2Real scope with interesting results. Furthermore, it covers the application of policy transfer between different experiments, describing the impact of competing environments on RL agents performance. RL and deep RL techniques are proven as profitable market maker approaches, leading to a better understanding of their behavior in stock markets.

Related papers

• Deep Reinforcement Learning Agents for Strategic Production Policies in Microeconomic Market Simulations [1.6499388997661122]
  We propose a DRL-based approach to obtain an effective policy in competitive markets with multiple producers. Our framework enables agents to learn adaptive production policies to several simulations that consistently outperform static and random strategies. The results show that agents trained with DRL can strategically adjust production levels to maximize long-term profitability.
  arXiv  Detail & Related papers  (2024-10-27T18:38:05Z)
• When AI Meets Finance (StockAgent): Large Language Model-based Stock Trading in Simulated Real-world Environments [55.19252983108372]
  We have developed a multi-agent AI system called StockAgent, driven by LLMs. The StockAgent allows users to evaluate the impact of different external factors on investor trading. It avoids the test set leakage issue present in existing trading simulation systems based on AI Agents.
  arXiv  Detail & Related papers  (2024-07-15T06:49:30Z)
• Reinforcement Learning in Agent-Based Market Simulation: Unveiling Realistic Stylized Facts and Behavior [7.8082951131908604]
  Investors and regulators can benefit from a realistic market simulator that enables them to anticipate the consequences of their decisions in real markets. Traditional rule-based market simulators often fall short in accurately capturing the dynamic behavior of market participants. In this study, we explore an agent-based simulation framework employing reinforcement learning (RL) agents.
  arXiv  Detail & Related papers  (2024-03-28T19:06:50Z)
• An Auction-based Marketplace for Model Trading in Federated Learning [54.79736037670377]
  Federated learning (FL) is increasingly recognized for its efficacy in training models using locally distributed data. We frame FL as a marketplace of models, where clients act as both buyers and sellers. We propose an auction-based solution to ensure proper pricing based on performance gain.
  arXiv  Detail & Related papers  (2024-02-02T07:25:53Z)
• Harnessing Deep Q-Learning for Enhanced Statistical Arbitrage in High-Frequency Trading: A Comprehensive Exploration [0.0]
  Reinforcement Learning (RL) is a branch of machine learning where agents learn by interacting with their environment. This paper dives deep into the integration of RL in statistical arbitrage strategies tailored for High-Frequency Trading (HFT) scenarios. Through extensive simulations and backtests, our research reveals that RL not only enhances the adaptability of trading strategies but also shows promise in improving profitability metrics and risk-adjusted returns.
  arXiv  Detail & Related papers  (2023-09-13T06:15:40Z)
• IMM: An Imitative Reinforcement Learning Approach with Predictive Representation Learning for Automatic Market Making [33.23156884634365]
  Reinforcement Learning technology has achieved remarkable success in quantitative trading. Most existing RL-based market making methods focus on optimizing single-price level strategies. We propose Imitative Market Maker (IMM), a novel RL framework leveraging both knowledge from suboptimal signal-based experts and direct policy interactions.
  arXiv  Detail & Related papers  (2023-08-17T11:04:09Z)
• Deep Reinforcement Learning Approach for Trading Automation in The Stock Market [0.0]
  This paper presents a model to generate profitable trades in the stock market using Deep Reinforcement Learning (DRL) algorithms. We formulate the trading problem as a Partially Observed Markov Decision Process (POMDP) model, considering the constraints imposed by the stock market. We then solve the formulated POMDP problem using the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm reporting a 2.68 Sharpe Ratio on unseen data set.
  arXiv  Detail & Related papers  (2022-07-05T11:34:29Z)
• FinRL-Meta: A Universe of Near-Real Market Environments for Data-Driven Deep Reinforcement Learning in Quantitative Finance [58.77314662664463]
  FinRL-Meta builds a universe of market environments for data-driven financial reinforcement learning. First, FinRL-Meta separates financial data processing from the design pipeline of DRL-based strategy. Second, FinRL-Meta provides hundreds of market environments for various trading tasks.
  arXiv  Detail & Related papers  (2021-12-13T16:03:37Z)
• OSOUM Framework for Trading Data Research [79.0383470835073]
  We supply, to the best of our knowledge, the first open source simulation platform, Open SOUrce Market Simulator (OSOUM) to analyze trading markets and specifically data markets. We describe and implement a specific data market model, consisting of two types of agents: sellers who own various datasets available for acquisition, and buyers searching for relevant and beneficial datasets for purchase. Although commercial frameworks, intended for handling data markets, already exist, we provide a free and extensive end-to-end research tool for simulating possible behavior for both buyers and sellers participating in (data) markets.
  arXiv  Detail & Related papers  (2021-02-18T09:20:26Z)
• Reinforcement-Learning based Portfolio Management with Augmented Asset Movement Prediction States [71.54651874063865]
  Portfolio management (PM) aims to achieve investment goals such as maximal profits or minimal risks. In this paper, we propose SARL, a novel State-Augmented RL framework for PM. Our framework aims to address two unique challenges in financial PM: (1) data Heterogeneous data -- the collected information for each asset is usually diverse, noisy and imbalanced (e.g., news articles); and (2) environment uncertainty -- the financial market is versatile and non-stationary.
  arXiv  Detail & Related papers  (2020-02-09T08:10:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.