Explainable Post hoc Portfolio Management Financial Policy of a Deep Reinforcement Learning agent

• URL: http://arxiv.org/abs/2407.14486v1
• Date: Fri, 19 Jul 2024 17:40:39 GMT
• Title: Explainable Post hoc Portfolio Management Financial Policy of a Deep Reinforcement Learning agent
• Authors: Alejandra de la Rica Escudero, Eduardo C. Garrido-Merchan, Maria Coronado-Vaca,
• Abstract summary: We develop a novel Explainable Deep Reinforcement Learning (XDRL) approach for portfolio management. By executing our methodology, we can interpret in prediction time the actions of the agent to assess whether they follow the requisites of an investment policy.
• Score: 44.99833362998488
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Financial portfolio management investment policies computed quantitatively by modern portfolio theory techniques like the Markowitz model rely on a set on assumptions that are not supported by data in high volatility markets. Hence, quantitative researchers are looking for alternative models to tackle this problem. Concretely, portfolio management is a problem that has been successfully addressed recently by Deep Reinforcement Learning (DRL) approaches. In particular, DRL algorithms train an agent by estimating the distribution of the expected reward of every action performed by an agent given any financial state in a simulator. However, these methods rely on Deep Neural Networks model to represent such a distribution, that although they are universal approximator models, they cannot explain its behaviour, given by a set of parameters that are not interpretable. Critically, financial investors policies require predictions to be interpretable, so DRL agents are not suited to follow a particular policy or explain their actions. In this work, we developed a novel Explainable Deep Reinforcement Learning (XDRL) approach for portfolio management, integrating the Proximal Policy Optimization (PPO) with the model agnostic explainable techniques of feature importance, SHAP and LIME to enhance transparency in prediction time. By executing our methodology, we can interpret in prediction time the actions of the agent to assess whether they follow the requisites of an investment policy or to assess the risk of following the agent suggestions. To the best of our knowledge, our proposed approach is the first explainable post hoc portfolio management financial policy of a DRL agent. We empirically illustrate our methodology by successfully identifying key features influencing investment decisions, which demonstrate the ability to explain the agent actions in prediction time.

Related papers

• VinePPO: Unlocking RL Potential For LLM Reasoning Through Refined Credit Assignment [66.80143024475635]
  We propose VinePPO, a straightforward approach to compute unbiased Monte Carlo-based estimates. We show that VinePPO consistently outperforms PPO and other RL-free baselines across MATH and GSM8K datasets.
  arXiv  Detail & Related papers  (2024-10-02T15:49:30Z)
• When AI Meets Finance (StockAgent): Large Language Model-based Stock Trading in Simulated Real-world Environments [55.19252983108372]
  We have developed a multi-agent AI system called StockAgent, driven by LLMs. The StockAgent allows users to evaluate the impact of different external factors on investor trading. It avoids the test set leakage issue present in existing trading simulation systems based on AI Agents.
  arXiv  Detail & Related papers  (2024-07-15T06:49:30Z)
• Combining Transformer based Deep Reinforcement Learning with Black-Litterman Model for Portfolio Optimization [0.0]
  As a model-free algorithm, deep reinforcement learning (DRL) agent learns and makes decisions by interacting with the environment in an unsupervised way. We propose a hybrid portfolio optimization model combining the DRL agent and the Black-Litterman (BL) model. Our DRL agent significantly outperforms various comparison portfolio choice strategies and alternative DRL frameworks by at least 42% in terms of accumulated return.
  arXiv  Detail & Related papers  (2024-02-23T16:01:37Z)
• Diffusion Variational Autoencoder for Tackling Stochasticity in Multi-Step Regression Stock Price Prediction [54.21695754082441]
  Multi-step stock price prediction over a long-term horizon is crucial for forecasting its volatility. Current solutions to multi-step stock price prediction are mostly designed for single-step, classification-based predictions. We combine a deep hierarchical variational-autoencoder (VAE) and diffusion probabilistic techniques to do seq2seq stock prediction. Our model is shown to outperform state-of-the-art solutions in terms of its prediction accuracy and variance.
  arXiv  Detail & Related papers  (2023-08-18T16:21:15Z)
• Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models [51.3422222472898]
  We document the capability of large language models (LLMs) like ChatGPT to predict stock price movements using news headlines. We develop a theoretical model incorporating information capacity constraints, underreaction, limits-to-arbitrage, and LLMs.
  arXiv  Detail & Related papers  (2023-04-15T19:22:37Z)
• Deep Reinforcement Learning Approach for Trading Automation in The Stock Market [0.0]
  This paper presents a model to generate profitable trades in the stock market using Deep Reinforcement Learning (DRL) algorithms. We formulate the trading problem as a Partially Observed Markov Decision Process (POMDP) model, considering the constraints imposed by the stock market. We then solve the formulated POMDP problem using the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm reporting a 2.68 Sharpe Ratio on unseen data set.
  arXiv  Detail & Related papers  (2022-07-05T11:34:29Z)
• Explainable Deep Reinforcement Learning for Portfolio Management: An Empirical Approach [30.283740528236752]
  It is challenging to understand a DRL-based trading strategy because of the black-box nature of deep neural networks. We propose an empirical approach to explain the strategies of DRL agents for the portfolio management task.
  arXiv  Detail & Related papers  (2021-11-07T04:23:48Z)
• Learning Risk Preferences from Investment Portfolios Using Inverse Optimization [25.19470942583387]
  This paper presents a novel approach of measuring risk preference from existing portfolios using inverse optimization. We demonstrate our methods on real market data that consists of 20 years of asset pricing and 10 years of mutual fund portfolio holdings.
  arXiv  Detail & Related papers  (2020-10-04T21:29:29Z)
• Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observations [88.94162416324505]
  A deep reinforcement learning (DRL) agent observes its states through observations, which may contain natural measurement errors or adversarial noises. Since the observations deviate from the true states, they can mislead the agent into making suboptimal actions. We show that naively applying existing techniques on improving robustness for classification tasks, like adversarial training, is ineffective for many RL tasks.
  arXiv  Detail & Related papers  (2020-03-19T17:59:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.