Reinforcement Learning with Maskable Stock Representation for Portfolio Management in Customizable Stock Pools

• URL: http://arxiv.org/abs/2311.10801v4
• Date: Tue, 27 Feb 2024 08:08:03 GMT
• Title: Reinforcement Learning with Maskable Stock Representation for Portfolio Management in Customizable Stock Pools
• Authors: Wentao Zhang, Yilei Zhao, Shuo Sun, Jie Ying, Yonggang Xie, Zitao Song, Xinrun Wang, Bo An
• Abstract summary: Portfolio management (PM) is a fundamental financial trading task, which explores the optimal periodical reallocation of capitals into different stocks to pursue long-term profits. ExistingReinforcement learning (RL) methods require to retrain RL agents even with a tiny change of the stock pool, which leads to high computational cost and unstable performance. We propose EarnMore to handle PM with CSPs through one-shot training in a global stock pool.
• Score: 34.97636568457075
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Portfolio management (PM) is a fundamental financial trading task, which explores the optimal periodical reallocation of capitals into different stocks to pursue long-term profits. Reinforcement learning (RL) has recently shown its potential to train profitable agents for PM through interacting with financial markets. However, existing work mostly focuses on fixed stock pools, which is inconsistent with investors' practical demand. Specifically, the target stock pool of different investors varies dramatically due to their discrepancy on market states and individual investors may temporally adjust stocks they desire to trade (e.g., adding one popular stocks), which lead to customizable stock pools (CSPs). Existing RL methods require to retrain RL agents even with a tiny change of the stock pool, which leads to high computational cost and unstable performance. To tackle this challenge, we propose EarnMore, a rEinforcement leARNing framework with Maskable stOck REpresentation to handle PM with CSPs through one-shot training in a global stock pool (GSP). Specifically, we first introduce a mechanism to mask out the representation of the stocks outside the target pool. Second, we learn meaningful stock representations through a self-supervised masking and reconstruction process. Third, a re-weighting mechanism is designed to make the portfolio concentrate on favorable stocks and neglect the stocks outside the target pool. Through extensive experiments on 8 subset stock pools of the US stock market, we demonstrate that EarnMore significantly outperforms 14 state-of-the-art baselines in terms of 6 popular financial metrics with over 40% improvement on profit.

Related papers

• VinePPO: Unlocking RL Potential For LLM Reasoning Through Refined Credit Assignment [66.80143024475635]
  We propose VinePPO, a straightforward approach to compute unbiased Monte Carlo-based estimates. We show that VinePPO consistently outperforms PPO and other RL-free baselines across MATH and GSM8K datasets.
  arXiv  Detail & Related papers  (2024-10-02T15:49:30Z)
• Portfolio Management using Deep Reinforcement Learning [0.0]
  We propose a reinforced portfolio manager offering assistance in the allocation of weights to assets. The environment proffers the manager the freedom to go long and even short on the assets. The manager performs financial transactions in a postulated liquid market without any transaction charges.
  arXiv  Detail & Related papers  (2024-05-01T22:28:55Z)
• Diffusion Variational Autoencoder for Tackling Stochasticity in Multi-Step Regression Stock Price Prediction [54.21695754082441]
  Multi-step stock price prediction over a long-term horizon is crucial for forecasting its volatility. Current solutions to multi-step stock price prediction are mostly designed for single-step, classification-based predictions. We combine a deep hierarchical variational-autoencoder (VAE) and diffusion probabilistic techniques to do seq2seq stock prediction. Our model is shown to outperform state-of-the-art solutions in terms of its prediction accuracy and variance.
  arXiv  Detail & Related papers  (2023-08-18T16:21:15Z)
• Asset Allocation: From Markowitz to Deep Reinforcement Learning [2.0305676256390934]
  Asset allocation is an investment strategy that aims to balance risk and reward by constantly redistributing the portfolio's assets. We conduct an extensive benchmark study to determine the efficacy and reliability of a number of optimization techniques.
  arXiv  Detail & Related papers  (2022-07-14T14:44:04Z)
• Quantitative Stock Investment by Routing Uncertainty-Aware Trading Experts: A Multi-Task Learning Approach [29.706515133374193]
  We show that existing deep learning methods are sensitive to random seeds and network routers. We propose a novel two-stage mixture-of-experts (MoE) framework for quantitative investment to mimic the efficient bottom-up trading strategy design workflow of successful trading firms. AlphaMix significantly outperforms many state-of-the-art baselines in terms of four financial criteria.
  arXiv  Detail & Related papers  (2022-06-07T08:58:00Z)
• Deep Q-Learning Market Makers in a Multi-Agent Simulated Stock Market [58.720142291102135]
  This paper focuses precisely on the study of these markets makers strategies from an agent-based perspective. We propose the application of Reinforcement Learning (RL) for the creation of intelligent market markers in simulated stock markets.
  arXiv  Detail & Related papers  (2021-12-08T14:55:21Z)
• Deep Stock Trading: A Hierarchical Reinforcement Learning Framework for Portfolio Optimization and Order Execution [26.698261314897195]
  We propose a hierarchical reinforced stock trading system for portfolio management (HRPM) We decompose the trading process into a hierarchy of portfolio management over trade execution and train the corresponding policies. HRPM achieves significant improvement against many state-of-the-art approaches.
  arXiv  Detail & Related papers  (2020-12-23T12:09:26Z)
• MAPS: Multi-agent Reinforcement Learning-based Portfolio Management System [23.657021288146158]
  We propose the Multi-Agent reinforcement learning-based Portfolio management System (MAPS) MAPS is a cooperative system in which each agent is an independent "investor" creating its own portfolio. Experiment results with 12 years of US market data show that MAPS outperforms most of the baselines in terms of Sharpe ratio.
  arXiv  Detail & Related papers  (2020-07-10T14:08:12Z)
• Deep Stock Predictions [58.720142291102135]
  We consider the design of a trading strategy that performs portfolio optimization using Long Short Term Memory (LSTM) neural networks. We then customize the loss function used to train the LSTM to increase the profit earned. We find the LSTM model with the customized loss function to have an improved performance in the training bot over a regressive baseline such as ARIMA.
  arXiv  Detail & Related papers  (2020-06-08T23:37:47Z)
• Reinforcement-Learning based Portfolio Management with Augmented Asset Movement Prediction States [71.54651874063865]
  Portfolio management (PM) aims to achieve investment goals such as maximal profits or minimal risks. In this paper, we propose SARL, a novel State-Augmented RL framework for PM. Our framework aims to address two unique challenges in financial PM: (1) data Heterogeneous data -- the collected information for each asset is usually diverse, noisy and imbalanced (e.g., news articles); and (2) environment uncertainty -- the financial market is versatile and non-stationary.
  arXiv  Detail & Related papers  (2020-02-09T08:10:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.