Combining Transformer based Deep Reinforcement Learning with Black-Litterman Model for Portfolio Optimization

• URL: http://arxiv.org/abs/2402.16609v1
• Date: Fri, 23 Feb 2024 16:01:37 GMT
• Title: Combining Transformer based Deep Reinforcement Learning with Black-Litterman Model for Portfolio Optimization
• Authors: Ruoyu Sun (1), Angelos Stefanidis (2), Zhengyong Jiang (2), Jionglong Su (2) ((1) Xi'an Jiaotong-Liverpool University, School of Mathematics and Physics, Department of Financial and Actuarial Mathematics (2) Xi'an Jiaotong-Liverpool University Entrepreneur College (Taicang), School of AI and Advanced Computing (1))
• Abstract summary: As a model-free algorithm, deep reinforcement learning (DRL) agent learns and makes decisions by interacting with the environment in an unsupervised way. We propose a hybrid portfolio optimization model combining the DRL agent and the Black-Litterman (BL) model. Our DRL agent significantly outperforms various comparison portfolio choice strategies and alternative DRL frameworks by at least 42% in terms of accumulated return.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As a model-free algorithm, deep reinforcement learning (DRL) agent learns and makes decisions by interacting with the environment in an unsupervised way. In recent years, DRL algorithms have been widely applied by scholars for portfolio optimization in consecutive trading periods, since the DRL agent can dynamically adapt to market changes and does not rely on the specification of the joint dynamics across the assets. However, typical DRL agents for portfolio optimization cannot learn a policy that is aware of the dynamic correlation between portfolio asset returns. Since the dynamic correlations among portfolio assets are crucial in optimizing the portfolio, the lack of such knowledge makes it difficult for the DRL agent to maximize the return per unit of risk, especially when the target market permits short selling (i.e., the US stock market). In this research, we propose a hybrid portfolio optimization model combining the DRL agent and the Black-Litterman (BL) model to enable the DRL agent to learn the dynamic correlation between the portfolio asset returns and implement an efficacious long/short strategy based on the correlation. Essentially, the DRL agent is trained to learn the policy to apply the BL model to determine the target portfolio weights. To test our DRL agent, we construct the portfolio based on all the Dow Jones Industrial Average constitute stocks. Empirical results of the experiments conducted on real-world United States stock market data demonstrate that our DRL agent significantly outperforms various comparison portfolio choice strategies and alternative DRL frameworks by at least 42% in terms of accumulated return. In terms of the return per unit of risk, our DRL agent significantly outperforms various comparative portfolio choice strategies and alternative strategies based on other machine learning frameworks.

Related papers

• Bridging and Modeling Correlations in Pairwise Data for Direct Preference Optimization [75.1240295759264]
  We propose an effective framework for Bridging and Modeling Correlations in pairwise data, named BMC. We increase the consistency and informativeness of the pairwise preference signals through targeted modifications. We identify that DPO alone is insufficient to model these correlations and capture nuanced variations.
  arXiv  Detail & Related papers  (2024-08-14T11:29:47Z)
• Optimizing Portfolio with Two-Sided Transactions and Lending: A Reinforcement Learning Framework [0.0]
  This study presents a Reinforcement Learning-based portfolio management model tailored for high-risk environments. We implement the model using the Soft Actor-Critic (SAC) agent with a Convolutional Neural Network with Multi-Head Attention. Tested over two 16-month periods of varying market volatility, the model significantly outperformed benchmarks.
  arXiv  Detail & Related papers  (2024-08-09T23:36:58Z)
• Hedge Fund Portfolio Construction Using PolyModel Theory and iTransformer [1.4061979259370274]
  We implement the PolyModel theory for constructing a hedge fund portfolio. We create quantitative measures such as Long-term Alpha, Long-term Ratio, and SVaR. We also employ the latest deep learning techniques (iTransformer) to capture the upward trend.
  arXiv  Detail & Related papers  (2024-08-06T17:55:58Z)
• Explainable Post hoc Portfolio Management Financial Policy of a Deep Reinforcement Learning agent [44.99833362998488]
  We develop a novel Explainable Deep Reinforcement Learning (XDRL) approach for portfolio management. By executing our methodology, we can interpret in prediction time the actions of the agent to assess whether they follow the requisites of an investment policy.
  arXiv  Detail & Related papers  (2024-07-19T17:40:39Z)
• When AI Meets Finance (StockAgent): Large Language Model-based Stock Trading in Simulated Real-world Environments [55.19252983108372]
  We have developed a multi-agent AI system called StockAgent, driven by LLMs. The StockAgent allows users to evaluate the impact of different external factors on investor trading. It avoids the test set leakage issue present in existing trading simulation systems based on AI Agents.
  arXiv  Detail & Related papers  (2024-07-15T06:49:30Z)
• Deep Reinforcement Learning and Mean-Variance Strategies for Responsible Portfolio Optimization [49.396692286192206]
  We study the use of deep reinforcement learning for responsible portfolio optimization by incorporating ESG states and objectives. Our results show that deep reinforcement learning policies can provide competitive performance against mean-variance approaches for responsible portfolio allocation.
  arXiv  Detail & Related papers  (2024-03-25T12:04:03Z)
• Data-Efficient Task Generalization via Probabilistic Model-based Meta Reinforcement Learning [58.575939354953526]
  PACOH-RL is a novel model-based Meta-Reinforcement Learning (Meta-RL) algorithm designed to efficiently adapt control policies to changing dynamics. Existing Meta-RL methods require abundant meta-learning data, limiting their applicability in settings such as robotics. Our experiment results demonstrate that PACOH-RL outperforms model-based RL and model-based Meta-RL baselines in adapting to new dynamic conditions.
  arXiv  Detail & Related papers  (2023-11-13T18:51:57Z)
• Mastering the Unsupervised Reinforcement Learning Benchmark from Pixels [112.63440666617494]
  Reinforcement learning algorithms can succeed but require large amounts of interactions between the agent and the environment. We propose a new method to solve it, using unsupervised model-based RL, for pre-training the agent. We show robust performance on the Real-Word RL benchmark, hinting at resiliency to environment perturbations during adaptation.
  arXiv  Detail & Related papers  (2022-09-24T14:22:29Z)
• Model-Free Reinforcement Learning for Asset Allocation [0.0]
  This study investigated the performance of reinforcement learning when applied to portfolio management using model-free deep RL agents. We trained several RL agents on real-world stock prices to learn how to perform asset allocation. Four RL agents (A2C, SAC, PPO, and TRPO) outperformed the best baseline, MPT, overall.
  arXiv  Detail & Related papers  (2022-09-21T16:00:24Z)
• Explainable Deep Reinforcement Learning for Portfolio Management: An Empirical Approach [30.283740528236752]
  It is challenging to understand a DRL-based trading strategy because of the black-box nature of deep neural networks. We propose an empirical approach to explain the strategies of DRL agents for the portfolio management task.
  arXiv  Detail & Related papers  (2021-11-07T04:23:48Z)
• A General Framework on Enhancing Portfolio Management with Reinforcement Learning [3.6985496077087743]
  Portfolio management concerns continuous reallocation of funds and assets across financial instruments to meet the desired returns to risk profile. Deep reinforcement learning (RL) has gained increasing interest in portfolio management, where RL agents are trained base on financial data to optimize the asset reallocation process. We propose a general RL framework for asset management that enables continuous asset weights, short selling and making decisions with relevant features.
  arXiv  Detail & Related papers  (2019-11-26T23:41:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.