Markowitz Meets Bellman: Knowledge-distilled Reinforcement Learning for Portfolio Management

• URL: http://arxiv.org/abs/2405.05449v1
• Date: Wed, 8 May 2024 22:54:04 GMT
• Title: Markowitz Meets Bellman: Knowledge-distilled Reinforcement Learning for Portfolio Management
• Authors: Gang Hu, Ming Gu,
• Abstract summary: This paper introduces a hybrid approach combining Markowitz's portfolio theory with reinforcement learning. In particular, our proposed method, called KDD (Knowledge Distillation DDPG), consist of two training stages: supervised and reinforcement learning stages. A comparative analysis against standard financial models and AI frameworks, using metrics like returns, the Sharpe ratio, and nine evaluation indices, reveals our model's superiority.
• Score: 7.199922073535674
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Investment portfolios, central to finance, balance potential returns and risks. This paper introduces a hybrid approach combining Markowitz's portfolio theory with reinforcement learning, utilizing knowledge distillation for training agents. In particular, our proposed method, called KDD (Knowledge Distillation DDPG), consist of two training stages: supervised and reinforcement learning stages. The trained agents optimize portfolio assembly. A comparative analysis against standard financial models and AI frameworks, using metrics like returns, the Sharpe ratio, and nine evaluation indices, reveals our model's superiority. It notably achieves the highest yield and Sharpe ratio of 2.03, ensuring top profitability with the lowest risk in comparable return scenarios.

Related papers

• Optimizing Portfolio with Two-Sided Transactions and Lending: A Reinforcement Learning Framework [0.0]
  This study presents a Reinforcement Learning-based portfolio management model tailored for high-risk environments. We implement the model using the Soft Actor-Critic (SAC) agent with a Convolutional Neural Network with Multi-Head Attention. Tested over two 16-month periods of varying market volatility, the model significantly outperformed benchmarks.
  arXiv  Detail & Related papers  (2024-08-09T23:36:58Z)
• Deep Reinforcement Learning and Mean-Variance Strategies for Responsible Portfolio Optimization [49.396692286192206]
  We study the use of deep reinforcement learning for responsible portfolio optimization by incorporating ESG states and objectives. Our results show that deep reinforcement learning policies can provide competitive performance against mean-variance approaches for responsible portfolio allocation.
  arXiv  Detail & Related papers  (2024-03-25T12:04:03Z)
• Cryptocurrency Portfolio Optimization by Neural Networks [81.20955733184398]
  This paper proposes an effective algorithm based on neural networks to take advantage of these investment products. A deep neural network, which outputs the allocation weight of each asset at a time interval, is trained to maximize the Sharpe ratio. A novel loss term is proposed to regulate the network's bias towards a specific asset, thus enforcing the network to learn an allocation strategy that is close to a minimum variance strategy.
  arXiv  Detail & Related papers  (2023-10-02T12:33:28Z)
• A Comparative Analysis of Portfolio Optimization Using Mean-Variance, Hierarchical Risk Parity, and Reinforcement Learning Approaches on the Indian Stock Market [0.0]
  This paper presents a comparative analysis of the performances of three portfolio optimization approaches. The portfolios are trained and tested over several stock data and their performances are compared on their annual returns, annual risks, and Sharpe ratios.
  arXiv  Detail & Related papers  (2023-05-27T16:38:18Z)
• Can Perturbations Help Reduce Investment Risks? Risk-Aware Stock Recommendation via Split Variational Adversarial Training [44.7991257631318]
  We propose a novel Split Variational Adrial Training (SVAT) method for risk-aware stock recommendation. By lowering the volatility of the stock recommendation model, SVAT effectively reduces investment risks and outperforms state-of-the-art baselines by more than 30% in terms of risk-adjusted profits.
  arXiv  Detail & Related papers  (2023-04-20T12:10:12Z)
• Factor Investing with a Deep Multi-Factor Model [123.52358449455231]
  We develop a novel deep multi-factor model that adopts industry neutralization and market neutralization modules with clear financial insights. Tests on real-world stock market data demonstrate the effectiveness of our deep multi-factor model.
  arXiv  Detail & Related papers  (2022-10-22T14:47:11Z)
• Asset Allocation: From Markowitz to Deep Reinforcement Learning [2.0305676256390934]
  Asset allocation is an investment strategy that aims to balance risk and reward by constantly redistributing the portfolio's assets. We conduct an extensive benchmark study to determine the efficacy and reliability of a number of optimization techniques.
  arXiv  Detail & Related papers  (2022-07-14T14:44:04Z)
• Deep Risk Model: A Deep Learning Solution for Mining Latent Risk Factors to Improve Covariance Matrix Estimation [8.617532047238461]
  We propose a deep learning solution to effectively "design" risk factors with neural networks. Our method can obtain $1.9%$ higher explained variance measured by $R2$ and also reduce the risk of a global minimum variance portfolio.
  arXiv  Detail & Related papers  (2021-07-12T05:30:50Z)
• Softmax with Regularization: Better Value Estimation in Multi-Agent Reinforcement Learning [72.28520951105207]
  Overestimation in $Q$-learning is an important problem that has been extensively studied in single-agent reinforcement learning. We propose a novel regularization-based update scheme that penalizes large joint action-values deviating from a baseline. We show that our method provides a consistent performance improvement on a set of challenging StarCraft II micromanagement tasks.
  arXiv  Detail & Related papers  (2021-03-22T14:18:39Z)
• Stock2Vec: A Hybrid Deep Learning Framework for Stock Market Prediction with Representation Learning and Temporal Convolutional Network [71.25144476293507]
  We have proposed to develop a global hybrid deep learning framework to predict the daily prices in the stock market. With representation learning, we derived an embedding called Stock2Vec, which gives us insight for the relationship among different stocks. Our hybrid framework integrates both advantages and achieves better performance on the stock price prediction task than several popular benchmarked models.
  arXiv  Detail & Related papers  (2020-09-29T22:54:30Z)
• MAPS: Multi-agent Reinforcement Learning-based Portfolio Management System [23.657021288146158]
  We propose the Multi-Agent reinforcement learning-based Portfolio management System (MAPS) MAPS is a cooperative system in which each agent is an independent "investor" creating its own portfolio. Experiment results with 12 years of US market data show that MAPS outperforms most of the baselines in terms of Sharpe ratio.
  arXiv  Detail & Related papers  (2020-07-10T14:08:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.