Deep Reinforcement Learning and Mean-Variance Strategies for Responsible Portfolio Optimization

• URL: http://arxiv.org/abs/2403.16667v1
• Date: Mon, 25 Mar 2024 12:04:03 GMT
• Title: Deep Reinforcement Learning and Mean-Variance Strategies for Responsible Portfolio Optimization
• Authors: Fernando Acero, Parisa Zehtabi, Nicolas Marchesotti, Michael Cashmore, Daniele Magazzeni, Manuela Veloso,
• Abstract summary: We study the use of deep reinforcement learning for responsible portfolio optimization by incorporating ESG states and objectives. Our results show that deep reinforcement learning policies can provide competitive performance against mean-variance approaches for responsible portfolio allocation.
• Score: 49.396692286192206
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Portfolio optimization involves determining the optimal allocation of portfolio assets in order to maximize a given investment objective. Traditionally, some form of mean-variance optimization is used with the aim of maximizing returns while minimizing risk, however, more recently, deep reinforcement learning formulations have been explored. Increasingly, investors have demonstrated an interest in incorporating ESG objectives when making investment decisions, and modifications to the classical mean-variance optimization framework have been developed. In this work, we study the use of deep reinforcement learning for responsible portfolio optimization, by incorporating ESG states and objectives, and provide comparisons against modified mean-variance approaches. Our results show that deep reinforcement learning policies can provide competitive performance against mean-variance approaches for responsible portfolio allocation across additive and multiplicative utility functions of financial and ESG responsibility objectives.

Related papers

• Quantum-Inspired Portfolio Optimization In The QUBO Framework [0.0]
  A quantum-inspired optimization approach is proposed to study the portfolio optimization aimed at maximizing the returns of investment portfolio while minimizing its risk by diversifying investment across different asset classes. This research contributes to the growing body of literature on quantum-inspired techniques in finance, demonstrating its potential as a powerful tool for asset allocation and portfolio management.
  arXiv  Detail & Related papers  (2024-10-08T11:36:43Z)
• Anatomy of Machines for Markowitz: Decision-Focused Learning for Mean-Variance Portfolio Optimization [27.791742749950203]
  Decision-Focused Learning can integrate prediction and optimization to improve decision-making outcomes. MSE treats the errors of all assets equally, but how does DFL reduce errors of different assets differently? This study aims to investigate how DFL adjusts stock return prediction models to optimize decisions in MVO.
  arXiv  Detail & Related papers  (2024-09-15T10:37:11Z)
• Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer [52.09480867526656]
  We identify the source of misalignment as a form of distributional shift and uncertainty in learning human preferences. To mitigate overoptimization, we first propose a theoretical algorithm that chooses the best policy for an adversarially chosen reward model. Using the equivalence between reward models and the corresponding optimal policy, the algorithm features a simple objective that combines a preference optimization loss and a supervised learning loss.
  arXiv  Detail & Related papers  (2024-05-26T05:38:50Z)
• Overcoming Reward Overoptimization via Adversarial Policy Optimization with Lightweight Uncertainty Estimation [46.61909578101735]
  Adversarial Policy Optimization (AdvPO) is a novel solution to the pervasive issue of reward over-optimization in Reinforcement Learning from Human Feedback. In this paper, we introduce a lightweight way to quantify uncertainties in rewards, relying solely on the last layer embeddings of the reward model.
  arXiv  Detail & Related papers  (2024-03-08T09:20:12Z)
• Unleashing the Potential of Large Language Models as Prompt Optimizers: An Analogical Analysis with Gradient-based Model Optimizers [108.72225067368592]
  We propose a novel perspective to investigate the design of large language models (LLMs)-based prompts. We identify two pivotal factors in model parameter learning: update direction and update method. In particular, we borrow the theoretical framework and learning methods from gradient-based optimization to design improved strategies.
  arXiv  Detail & Related papers  (2024-02-27T15:05:32Z)
• Causal Inference on Investment Constraints and Non-stationarity in Dynamic Portfolio Optimization through Reinforcement Learning [0.0]
  We have developed a dynamic asset allocation investment strategy using reinforcement learning techniques. We have addressed the crucial issue of incorporating non-stationarity of financial time series data into reinforcement learning algorithms. The application of reinforcement learning in investment strategies provides a remarkable advantage of setting the optimization problem flexibly.
  arXiv  Detail & Related papers  (2023-11-08T07:55:51Z)
• Acceleration in Policy Optimization [50.323182853069184]
  We work towards a unifying paradigm for accelerating policy optimization methods in reinforcement learning (RL) by integrating foresight in the policy improvement step via optimistic and adaptive updates. We define optimism as predictive modelling of the future behavior of a policy, and adaptivity as taking immediate and anticipatory corrective actions to mitigate errors from overshooting predictions or delayed responses to change. We design an optimistic policy gradient algorithm, adaptive via meta-gradient learning, and empirically highlight several design choices pertaining to acceleration, in an illustrative task.
  arXiv  Detail & Related papers  (2023-06-18T15:50:57Z)
• Bayesian Optimization of ESG Financial Investments [0.0]
  ESG (Economic, Social and Governance) criteria have become more significant in finance. This paper combines mathematical modelling, with ESG and finance.
  arXiv  Detail & Related papers  (2023-02-10T15:17:36Z)
• Asset Allocation: From Markowitz to Deep Reinforcement Learning [2.0305676256390934]
  Asset allocation is an investment strategy that aims to balance risk and reward by constantly redistributing the portfolio's assets. We conduct an extensive benchmark study to determine the efficacy and reliability of a number of optimization techniques.
  arXiv  Detail & Related papers  (2022-07-14T14:44:04Z)
• Policy Gradient Bayesian Robust Optimization for Imitation Learning [49.881386773269746]
  We derive a novel policy gradient-style robust optimization approach, PG-BROIL, to balance expected performance and risk. Results suggest PG-BROIL can produce a family of behaviors ranging from risk-neutral to risk-averse.
  arXiv  Detail & Related papers  (2021-06-11T16:49:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.