MacroHFT: Memory Augmented Context-aware Reinforcement Learning On High Frequency Trading

• URL: http://arxiv.org/abs/2406.14537v1
• Date: Thu, 20 Jun 2024 17:48:24 GMT
• Title: MacroHFT: Memory Augmented Context-aware Reinforcement Learning On High Frequency Trading
• Authors: Chuqiao Zong, Chaojie Wang, Molei Qin, Lei Feng, Xinrun Wang, Bo An,
• Abstract summary: Reinforcement learning (RL) has become another appealing approach for high-frequency trading (HFT) We propose a novel Memory Augmented Context-aware Reinforcement learning method On HFT, empha.k.a. MacroHFT. We show that MacroHFT can achieve state-of-the-art performance on minute-level trading tasks.
• Score: 20.3106468936159
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: High-frequency trading (HFT) that executes algorithmic trading in short time scales, has recently occupied the majority of cryptocurrency market. Besides traditional quantitative trading methods, reinforcement learning (RL) has become another appealing approach for HFT due to its terrific ability of handling high-dimensional financial data and solving sophisticated sequential decision-making problems, \emph{e.g.,} hierarchical reinforcement learning (HRL) has shown its promising performance on second-level HFT by training a router to select only one sub-agent from the agent pool to execute the current transaction. However, existing RL methods for HFT still have some defects: 1) standard RL-based trading agents suffer from the overfitting issue, preventing them from making effective policy adjustments based on financial context; 2) due to the rapid changes in market conditions, investment decisions made by an individual agent are usually one-sided and highly biased, which might lead to significant loss in extreme markets. To tackle these problems, we propose a novel Memory Augmented Context-aware Reinforcement learning method On HFT, \emph{a.k.a.} MacroHFT, which consists of two training phases: 1) we first train multiple types of sub-agents with the market data decomposed according to various financial indicators, specifically market trend and volatility, where each agent owns a conditional adapter to adjust its trading policy according to market conditions; 2) then we train a hyper-agent to mix the decisions from these sub-agents and output a consistently profitable meta-policy to handle rapid market fluctuations, equipped with a memory mechanism to enhance the capability of decision-making. Extensive experiments on various cryptocurrency markets demonstrate that MacroHFT can achieve state-of-the-art performance on minute-level trading tasks.

Related papers

• When AI Meets Finance (StockAgent): Large Language Model-based Stock Trading in Simulated Real-world Environments [55.19252983108372]
  We have developed a multi-agent AI system called StockAgent, driven by LLMs. The StockAgent allows users to evaluate the impact of different external factors on investor trading. It avoids the test set leakage issue present in existing trading simulation systems based on AI Agents.
  arXiv  Detail & Related papers  (2024-07-15T06:49:30Z)
• Can machine learning unlock new insights into high-frequency trading? [0.0]
  We introduce new metrics to identify liquidity-demanding and -supplying HFT strategies. Our metrics have implications for understanding the information production process in financial markets.
  arXiv  Detail & Related papers  (2024-05-13T18:28:39Z)
• Combining Deep Learning on Order Books with Reinforcement Learning for Profitable Trading [0.0]
  This work focuses on forecasting returns across multiple horizons using order flow and training three temporal-difference imbalance learning models for five financial instruments. The results prove potential but require further minimal modifications for consistently profitable trading to fully handle retail trading costs, slippage, and spread fluctuation.
  arXiv  Detail & Related papers  (2023-10-24T15:58:58Z)
• Diffusion Variational Autoencoder for Tackling Stochasticity in Multi-Step Regression Stock Price Prediction [54.21695754082441]
  Multi-step stock price prediction over a long-term horizon is crucial for forecasting its volatility. Current solutions to multi-step stock price prediction are mostly designed for single-step, classification-based predictions. We combine a deep hierarchical variational-autoencoder (VAE) and diffusion probabilistic techniques to do seq2seq stock prediction. Our model is shown to outperform state-of-the-art solutions in terms of its prediction accuracy and variance.
  arXiv  Detail & Related papers  (2023-08-18T16:21:15Z)
• IMM: An Imitative Reinforcement Learning Approach with Predictive Representation Learning for Automatic Market Making [33.23156884634365]
  Reinforcement Learning technology has achieved remarkable success in quantitative trading. Most existing RL-based market making methods focus on optimizing single-price level strategies. We propose Imitative Market Maker (IMM), a novel RL framework leveraging both knowledge from suboptimal signal-based experts and direct policy interactions.
  arXiv  Detail & Related papers  (2023-08-17T11:04:09Z)
• Factor Investing with a Deep Multi-Factor Model [123.52358449455231]
  We develop a novel deep multi-factor model that adopts industry neutralization and market neutralization modules with clear financial insights. Tests on real-world stock market data demonstrate the effectiveness of our deep multi-factor model.
  arXiv  Detail & Related papers  (2022-10-22T14:47:11Z)
• Deep Reinforcement Learning Approach for Trading Automation in The Stock Market [0.0]
  This paper presents a model to generate profitable trades in the stock market using Deep Reinforcement Learning (DRL) algorithms. We formulate the trading problem as a Partially Observed Markov Decision Process (POMDP) model, considering the constraints imposed by the stock market. We then solve the formulated POMDP problem using the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm reporting a 2.68 Sharpe Ratio on unseen data set.
  arXiv  Detail & Related papers  (2022-07-05T11:34:29Z)
• DeepScalper: A Risk-Aware Reinforcement Learning Framework to Capture Fleeting Intraday Trading Opportunities [33.28409845878758]
  We propose DeepScalper, a deep reinforcement learning framework for intraday trading. We show that DeepScalper significantly outperforms many state-of-the-art baselines in terms of four financial criteria.
  arXiv  Detail & Related papers  (2021-12-15T15:24:02Z)
• Deep Q-Learning Market Makers in a Multi-Agent Simulated Stock Market [58.720142291102135]
  This paper focuses precisely on the study of these markets makers strategies from an agent-based perspective. We propose the application of Reinforcement Learning (RL) for the creation of intelligent market markers in simulated stock markets.
  arXiv  Detail & Related papers  (2021-12-08T14:55:21Z)
• A Deep Reinforcement Learning Framework for Continuous Intraday Market Bidding [69.37299910149981]
  A key component for the successful renewable energy sources integration is the usage of energy storage. We propose a novel modelling framework for the strategic participation of energy storage in the European continuous intraday market. An distributed version of the fitted Q algorithm is chosen for solving this problem due to its sample efficiency. Results indicate that the agent converges to a policy that achieves in average higher total revenues than the benchmark strategy.
  arXiv  Detail & Related papers  (2020-04-13T13:50:13Z)
• Reinforcement-Learning based Portfolio Management with Augmented Asset Movement Prediction States [71.54651874063865]
  Portfolio management (PM) aims to achieve investment goals such as maximal profits or minimal risks. In this paper, we propose SARL, a novel State-Augmented RL framework for PM. Our framework aims to address two unique challenges in financial PM: (1) data Heterogeneous data -- the collected information for each asset is usually diverse, noisy and imbalanced (e.g., news articles); and (2) environment uncertainty -- the financial market is versatile and non-stationary.
  arXiv  Detail & Related papers  (2020-02-09T08:10:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.