IMM: An Imitative Reinforcement Learning Approach with Predictive
Representation Learning for Automatic Market Making
- URL: http://arxiv.org/abs/2308.08918v1
- Date: Thu, 17 Aug 2023 11:04:09 GMT
- Title: IMM: An Imitative Reinforcement Learning Approach with Predictive
Representation Learning for Automatic Market Making
- Authors: Hui Niu, Siyuan Li, Jiahao Zheng, Zhouchi Lin, Jian Li, Jian Guo, Bo
An
- Abstract summary: Reinforcement Learning technology has achieved remarkable success in quantitative trading.
Most existing RL-based market making methods focus on optimizing single-price level strategies.
We propose Imitative Market Maker (IMM), a novel RL framework leveraging both knowledge from suboptimal signal-based experts and direct policy interactions.
- Score: 33.23156884634365
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Market making (MM) has attracted significant attention in financial trading
owing to its essential function in ensuring market liquidity. With strong
capabilities in sequential decision-making, Reinforcement Learning (RL)
technology has achieved remarkable success in quantitative trading.
Nonetheless, most existing RL-based MM methods focus on optimizing single-price
level strategies which fail at frequent order cancellations and loss of queue
priority. Strategies involving multiple price levels align better with actual
trading scenarios. However, given the complexity that multi-price level
strategies involves a comprehensive trading action space, the challenge of
effectively training profitable RL agents for MM persists. Inspired by the
efficient workflow of professional human market makers, we propose Imitative
Market Maker (IMM), a novel RL framework leveraging both knowledge from
suboptimal signal-based experts and direct policy interactions to develop
multi-price level MM strategies efficiently. The framework start with
introducing effective state and action representations adept at encoding
information about multi-price level orders. Furthermore, IMM integrates a
representation learning unit capable of capturing both short- and long-term
market trends to mitigate adverse selection risk. Subsequently, IMM formulates
an expert strategy based on signals and trains the agent through the
integration of RL and imitation learning techniques, leading to efficient
learning. Extensive experimental results on four real-world market datasets
demonstrate that IMM outperforms current RL-based market making strategies in
terms of several financial criteria. The findings of the ablation study
substantiate the effectiveness of the model components.
Related papers
- Meta Reasoning for Large Language Models [58.87183757029041]
We introduce Meta-Reasoning Prompting (MRP), a novel and efficient system prompting method for large language models (LLMs)
MRP guides LLMs to dynamically select and apply different reasoning methods based on the specific requirements of each task.
We evaluate the effectiveness of MRP through comprehensive benchmarks.
arXiv Detail & Related papers (2024-06-17T16:14:11Z) - Developing A Multi-Agent and Self-Adaptive Framework with Deep
Reinforcement Learning for Dynamic Portfolio Risk Management [1.3505077405741583]
A multi-agent reinforcement learning (RL) approach is proposed to balance the trade-off between the overall portfolio returns and their potential risks.
The obtained empirical results clearly reveal the potential strengths of our proposed MASA framework.
arXiv Detail & Related papers (2024-02-01T11:31:26Z) - Harnessing Deep Q-Learning for Enhanced Statistical Arbitrage in
High-Frequency Trading: A Comprehensive Exploration [0.0]
Reinforcement Learning (RL) is a branch of machine learning where agents learn by interacting with their environment.
This paper dives deep into the integration of RL in statistical arbitrage strategies tailored for High-Frequency Trading (HFT) scenarios.
Through extensive simulations and backtests, our research reveals that RL not only enhances the adaptability of trading strategies but also shows promise in improving profitability metrics and risk-adjusted returns.
arXiv Detail & Related papers (2023-09-13T06:15:40Z) - Factor Investing with a Deep Multi-Factor Model [123.52358449455231]
We develop a novel deep multi-factor model that adopts industry neutralization and market neutralization modules with clear financial insights.
Tests on real-world stock market data demonstrate the effectiveness of our deep multi-factor model.
arXiv Detail & Related papers (2022-10-22T14:47:11Z) - MetaTrader: An Reinforcement Learning Approach Integrating Diverse
Policies for Portfolio Optimization [17.759687104376855]
We propose a novel two-stage-based approach for portfolio management.
In the first stage, incorporates an imitation learning into the reinforcement learning framework.
In the second stage, learns a meta-policy to recognize the market conditions and decide on the most proper learned policy to follow.
arXiv Detail & Related papers (2022-09-01T07:58:06Z) - Deep Reinforcement Learning Approach for Trading Automation in The Stock
Market [0.0]
This paper presents a model to generate profitable trades in the stock market using Deep Reinforcement Learning (DRL) algorithms.
We formulate the trading problem as a Partially Observed Markov Decision Process (POMDP) model, considering the constraints imposed by the stock market.
We then solve the formulated POMDP problem using the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm reporting a 2.68 Sharpe Ratio on unseen data set.
arXiv Detail & Related papers (2022-07-05T11:34:29Z) - Deep Q-Learning Market Makers in a Multi-Agent Simulated Stock Market [58.720142291102135]
This paper focuses precisely on the study of these markets makers strategies from an agent-based perspective.
We propose the application of Reinforcement Learning (RL) for the creation of intelligent market markers in simulated stock markets.
arXiv Detail & Related papers (2021-12-08T14:55:21Z) - Universal Trading for Order Execution with Oracle Policy Distillation [99.57416828489568]
We propose a novel universal trading policy optimization framework to bridge the gap between the noisy yet imperfect market states and the optimal action sequences for order execution.
We show that our framework can better guide the learning of the common policy towards practically optimal execution by an oracle teacher with perfect information.
arXiv Detail & Related papers (2021-01-28T05:52:18Z) - An Application of Deep Reinforcement Learning to Algorithmic Trading [4.523089386111081]
This scientific research paper presents an innovative approach based on deep reinforcement learning (DRL) to solve the algorithmic trading problem.
It proposes a novel DRL trading strategy so as to maximise the resulting Sharpe ratio performance indicator on a broad range of stock markets.
The training of the resulting reinforcement learning (RL) agent is entirely based on the generation of artificial trajectories from a limited set of stock market historical data.
arXiv Detail & Related papers (2020-04-07T14:57:23Z) - Reinforcement-Learning based Portfolio Management with Augmented Asset
Movement Prediction States [71.54651874063865]
Portfolio management (PM) aims to achieve investment goals such as maximal profits or minimal risks.
In this paper, we propose SARL, a novel State-Augmented RL framework for PM.
Our framework aims to address two unique challenges in financial PM: (1) data Heterogeneous data -- the collected information for each asset is usually diverse, noisy and imbalanced (e.g., news articles); and (2) environment uncertainty -- the financial market is versatile and non-stationary.
arXiv Detail & Related papers (2020-02-09T08:10:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.