Reinforcement Learning with Expert Trajectory For Quantitative Trading

• URL: http://arxiv.org/abs/2105.03844v1
• Date: Sun, 9 May 2021 05:49:21 GMT
• Title: Reinforcement Learning with Expert Trajectory For Quantitative Trading
• Authors: Sihang Chen, Weiqi Luo and Chao Yu
• Abstract summary: We model the price prediction problem as a Markov decision process (MDP), and optimize it by reinforcement learning with expert trajectory. We employ more than 100 short-term alpha factors instead of price, volume and several technical factors in used existing methods to describe the states of MDP. Experimental results evaluated on share price index futures in China, including IF (CSI 300) and IC (CSI 500)
• Score: 11.460285913081346
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, quantitative investment methods combined with artificial intelligence have attracted more and more attention from investors and researchers. Existing related methods based on the supervised learning are not very suitable for learning problems with long-term goals and delayed rewards in real futures trading. In this paper, therefore, we model the price prediction problem as a Markov decision process (MDP), and optimize it by reinforcement learning with expert trajectory. In the proposed method, we employ more than 100 short-term alpha factors instead of price, volume and several technical factors in used existing methods to describe the states of MDP. Furthermore, unlike DQN (deep Q-learning) and BC (behavior cloning) in related methods, we introduce expert experience in training stage, and consider both the expert-environment interaction and the agent-environment interaction to design the temporal difference error so that the agents are more adaptable for inevitable noise in financial data. Experimental results evaluated on share price index futures in China, including IF (CSI 300) and IC (CSI 500), show that the advantages of the proposed method compared with three typical technical analysis and two deep leaning based methods.

Related papers

• MR-Ben: A Meta-Reasoning Benchmark for Evaluating System-2 Thinking in LLMs [55.20845457594977]
  Large language models (LLMs) have shown increasing capability in problem-solving and decision-making. We present a process-based benchmark MR-Ben that demands a meta-reasoning skill. Our meta-reasoning paradigm is especially suited for system-2 slow thinking.
  arXiv  Detail & Related papers  (2024-06-20T03:50:23Z)
• A quantitative fusion strategy of stock picking and timing based on Particle Swarm Optimized-Back Propagation Neural Network and Multivariate Gaussian-Hidden Markov Model [0.0]
  This research introduces a pioneering quantitative fusion model combining stock timing and picking strategy. We conduct the prediction and trading on the basis of the screening stocks and stock market state outputted by MGHMM trained. Our fusion strategy incorporating stock picking and timing presented in this article provide a innovative technique for financial analysis.
  arXiv  Detail & Related papers  (2023-12-10T04:33:59Z)
• Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models [51.3422222472898]
  We document the capability of large language models (LLMs) like ChatGPT to predict stock price movements using news headlines. We develop a theoretical model incorporating information capacity constraints, underreaction, limits-to-arbitrage, and LLMs.
  arXiv  Detail & Related papers  (2023-04-15T19:22:37Z)
• Factor Investing with a Deep Multi-Factor Model [123.52358449455231]
  We develop a novel deep multi-factor model that adopts industry neutralization and market neutralization modules with clear financial insights. Tests on real-world stock market data demonstrate the effectiveness of our deep multi-factor model.
  arXiv  Detail & Related papers  (2022-10-22T14:47:11Z)
• Uncertainty-Driven Action Quality Assessment [67.20617610820857]
  We propose a novel probabilistic model, named Uncertainty-Driven AQA (UD-AQA), to capture the diversity among multiple judge scores. We generate the estimation of uncertainty for each prediction, which is employed to re-weight AQA regression loss. Our proposed method achieves competitive results on three benchmarks including the Olympic events MTL-AQA and FineDiving, and the surgical skill JIGSAWS datasets.
  arXiv  Detail & Related papers  (2022-07-29T07:21:15Z)
• Quantitative Stock Investment by Routing Uncertainty-Aware Trading Experts: A Multi-Task Learning Approach [29.706515133374193]
  We show that existing deep learning methods are sensitive to random seeds and network routers. We propose a novel two-stage mixture-of-experts (MoE) framework for quantitative investment to mimic the efficient bottom-up trading strategy design workflow of successful trading firms. AlphaMix significantly outperforms many state-of-the-art baselines in terms of four financial criteria.
  arXiv  Detail & Related papers  (2022-06-07T08:58:00Z)
• Bayesian Bilinear Neural Network for Predicting the Mid-price Dynamics in Limit-Order Book Markets [84.90242084523565]
  Traditional time-series econometric methods often appear incapable of capturing the true complexity of the multi-level interactions driving the price dynamics. By adopting a state-of-the-art second-order optimization algorithm, we train a Bayesian bilinear neural network with temporal attention. By addressing the use of predictive distributions to analyze errors and uncertainties associated with the estimated parameters and model forecasts, we thoroughly compare our Bayesian model with traditional ML alternatives.
  arXiv  Detail & Related papers  (2022-03-07T18:59:54Z)
• Deep Portfolio Optimization via Distributional Prediction of Residual Factors [3.9189409002585562]
  We propose a novel method of constructing a portfolio based on predicting the distribution of a financial quantity called residual factors. We demonstrate the efficacy of our method on U.S. and Japanese stock market data.
  arXiv  Detail & Related papers  (2020-12-14T04:09:52Z)
• Cross Learning in Deep Q-Networks [82.20059754270302]
  We propose a novel cross Q-learning algorithm, aim at alleviating the well-known overestimation problem in value-based reinforcement learning methods. Our algorithm builds on double Q-learning, by maintaining a set of parallel models and estimate the Q-value based on a randomly selected network.
  arXiv  Detail & Related papers  (2020-09-29T04:58:17Z)
• Bid Prediction in Repeated Auctions with Learning [30.07778295477907]
  We consider the problem of bid prediction in repeated auctions using a dataset from a mainstream sponsored search auction marketplace. We propose the use of no-regret based econometrics for bid prediction, modeling players as no-regret learners with respect to a utility function. We show that the no-regret econometric methods perform comparable to state-of-the-art time-series machine learning methods.
  arXiv  Detail & Related papers  (2020-07-26T18:14:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.