Deep Reinforcement Learning for Active High Frequency Trading

• URL: http://arxiv.org/abs/2101.07107v3
• Date: Sat, 19 Aug 2023 08:10:38 GMT
• Title: Deep Reinforcement Learning for Active High Frequency Trading
• Authors: Antonio Briola, Jeremy Turiel, Riccardo Marcaccioli, Alvaro Cauderan, Tomaso Aste
• Abstract summary: We introduce the first end-to-end Deep Reinforcement Learning (DRL) based framework for active high frequency trading in the stock market. We train DRL agents to trade one unit of Intel Corporation stock by employing the Proximal Policy Optimization algorithm.
• Score: 1.6874375111244329
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We introduce the first end-to-end Deep Reinforcement Learning (DRL) based framework for active high frequency trading in the stock market. We train DRL agents to trade one unit of Intel Corporation stock by employing the Proximal Policy Optimization algorithm. The training is performed on three contiguous months of high frequency Limit Order Book data, of which the last month constitutes the validation data. In order to maximise the signal to noise ratio in the training data, we compose the latter by only selecting training samples with largest price changes. The test is then carried out on the following month of data. Hyperparameters are tuned using the Sequential Model Based Optimization technique. We consider three different state characterizations, which differ in their LOB-based meta-features. Analysing the agents' performances on test data, we argue that the agents are able to create a dynamic representation of the underlying environment. They identify occasional regularities present in the data and exploit them to create long-term profitable trading strategies. Indeed, agents learn trading strategies able to produce stable positive returns in spite of the highly stochastic and non-stationary environment.

Related papers

• An Extremely Data-efficient and Generative LLM-based Reinforcement Learning Agent for Recommenders [1.0154385852423122]
  reinforcement learning (RL) algorithms have been instrumental in maximizing long-term customer satisfaction and avoiding short-term, myopic goals in industrial recommender systems. The goal is to train an RL agent to maximize the purchase reward given a detailed human instruction describing a desired product. This report also evaluates the RL agents trained using generative trajectories.
  arXiv  Detail & Related papers  (2024-08-28T10:31:50Z)
• Optimizing Deep Reinforcement Learning for American Put Option Hedging [0.0]
  This paper contributes to the existing literature on hedging American options with Deep Reinforcement Learning (DRL) Results highlight the importance of avoiding certain combinations, such as high learning rates with a high number of training episodes or low learning rates with few training episodes. This paper demonstrates that both single-train and weekly-train DRL agents outperform the Black-Scholes Delta method at transaction costs of 1% and 3%.
  arXiv  Detail & Related papers  (2024-05-14T13:41:44Z)
• Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learning [55.96599486604344]
  We introduce an approach aimed at enhancing the reasoning capabilities of Large Language Models (LLMs) through an iterative preference learning process. We use Monte Carlo Tree Search (MCTS) to iteratively collect preference data, utilizing its look-ahead ability to break down instance-level rewards into more granular step-level signals. The proposed algorithm employs Direct Preference Optimization (DPO) to update the LLM policy using this newly generated step-level preference data.
  arXiv  Detail & Related papers  (2024-05-01T11:10:24Z)
• Take the Bull by the Horns: Hard Sample-Reweighted Continual Training Improves LLM Generalization [165.98557106089777]
  A key challenge is to enhance the capabilities of large language models (LLMs) amid a looming shortage of high-quality training data. Our study starts from an empirical strategy for the light continual training of LLMs using their original pre-training data sets. We then formalize this strategy into a principled framework of Instance-Reweighted Distributionally Robust Optimization.
  arXiv  Detail & Related papers  (2024-02-22T04:10:57Z)
• How to Train Data-Efficient LLMs [56.41105687693619]
  We study data-efficient approaches for pre-training language models (LLMs) We find that Ask-LLM and Density sampling are the best methods in their respective categories. In our comparison of 19 samplers, involving hundreds of evaluation tasks and pre-training runs, we find that Ask-LLM and Density are the best methods in their respective categories.
  arXiv  Detail & Related papers  (2024-02-15T02:27:57Z)
• Asynchronous Deep Double Duelling Q-Learning for Trading-Signal Execution in Limit Order Book Markets [5.202524136984542]
  We employ deep reinforcement learning to train an agent to translate a high-frequency trading signal into a trading strategy that places individual limit orders. Based on the ABIDES limit order book simulator, we build a reinforcement learning OpenAI gym environment. We find that the RL agent learns an effective trading strategy for inventory management and order placing that outperforms a benchmark trading strategy having access to the same signal.
  arXiv  Detail & Related papers  (2023-01-20T17:19:18Z)
• Augmented Bilinear Network for Incremental Multi-Stock Time-Series Classification [83.23129279407271]
  We propose a method to efficiently retain the knowledge available in a neural network pre-trained on a set of securities. In our method, the prior knowledge encoded in a pre-trained neural network is maintained by keeping existing connections fixed. This knowledge is adjusted for the new securities by a set of augmented connections, which are optimized using the new data.
  arXiv  Detail & Related papers  (2022-07-23T18:54:10Z)
• Retrieval-Augmented Reinforcement Learning [63.32076191982944]
  We train a network to map a dataset of past experiences to optimal behavior. The retrieval process is trained to retrieve information from the dataset that may be useful in the current context. We show that retrieval-augmented R2D2 learns significantly faster than the baseline R2D2 agent and achieves higher scores.
  arXiv  Detail & Related papers  (2022-02-17T02:44:05Z)
• DAGA: Data Augmentation with a Generation Approach for Low-resource Tagging Tasks [88.62288327934499]
  We propose a novel augmentation method with language models trained on the linearized labeled sentences. Our method is applicable to both supervised and semi-supervised settings.
  arXiv  Detail & Related papers  (2020-11-03T07:49:15Z)
• A Deep Learning Framework for Predicting Digital Asset Price Movement from Trade-by-trade Data [20.392440676633573]
  This paper presents a framework that predicts price movement of cryptocurrencies from trade-by-trade data. The model is trained to achieve high performance on nearly a year of trade-by-trade data. In a realistic trading simulation setting, the prediction made by the model could be easily monetized.
  arXiv  Detail & Related papers  (2020-10-11T10:42:02Z)
• AAMDRL: Augmented Asset Management with Deep Reinforcement Learning [5.801876281373619]
  We show how Deep Reinforcement Learning can tackle this challenge. Our contributions are threefold: (i) the use of contextual information also referred to as augmented state in DRL, (ii) the impact of a one period lag between observations and actions, and (iii) the implementation of a new repetitive train test method called walk forward analysis. Although our experiment is on trading bots, it can easily be translated to other bot environments that operate in sequential environment with regime changes and noisy data.
  arXiv  Detail & Related papers  (2020-09-30T03:55:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.