Regret-Optimized Portfolio Enhancement through Deep Reinforcement Learning and Future Looking Rewards

• URL: http://arxiv.org/abs/2502.02619v1
• Date: Tue, 04 Feb 2025 11:45:59 GMT
• Title: Regret-Optimized Portfolio Enhancement through Deep Reinforcement Learning and Future Looking Rewards
• Authors: Daniil Karzanov, Rubén Garzón, Mikhail Terekhov, Caglar Gulcehre, Thomas Raffinot, Marcin Detyniecki,
• Abstract summary: This paper introduces a novel agent-based approach for enhancing existing portfolio strategies using Proximal Policy Optimization (PPO) Rather than focusing solely on traditional portfolio construction, our approach aims to improve an already high-performing strategy through dynamic rebalancing driven by PPO and Oracle agents.
• Score: 3.9795751586546766
• License:
• Abstract: This paper introduces a novel agent-based approach for enhancing existing portfolio strategies using Proximal Policy Optimization (PPO). Rather than focusing solely on traditional portfolio construction, our approach aims to improve an already high-performing strategy through dynamic rebalancing driven by PPO and Oracle agents. Our target is to enhance the traditional 60/40 benchmark (60% stocks, 40% bonds) by employing the Regret-based Sharpe reward function. To address the impact of transaction fee frictions and prevent signal loss, we develop a transaction cost scheduler. We introduce a future-looking reward function and employ synthetic data training through a circular block bootstrap method to facilitate the learning of generalizable allocation strategies. We focus on two key evaluation measures: return and maximum drawdown. Given the high stochasticity of financial markets, we train 20 independent agents each period and evaluate their average performance against the benchmark. Our method not only enhances the performance of the existing portfolio strategy through strategic rebalancing but also demonstrates strong results compared to other baselines.

Related papers

• Benchmarking Post-Training Quantization in LLMs: Comprehensive Taxonomy, Unified Evaluation, and Comparative Analysis [89.60263788590893]
  Post-training Quantization (PTQ) technique has been extensively adopted for large language models (LLMs) compression. Existing algorithms focus primarily on performance, overlooking the trade-off among model size, performance, and quantization bitwidth.
  arXiv  Detail & Related papers  (2025-02-18T07:35:35Z)
• From Novice to Expert: LLM Agent Policy Optimization via Step-wise Reinforcement Learning [62.54484062185869]
  We introduce StepAgent, which utilizes step-wise reward to optimize the agent's reinforcement learning process. We propose implicit-reward and inverse reinforcement learning techniques to facilitate agent reflection and policy adjustment.
  arXiv  Detail & Related papers  (2024-11-06T10:35:11Z)
• VinePPO: Unlocking RL Potential For LLM Reasoning Through Refined Credit Assignment [66.80143024475635]
  We propose VinePPO, a straightforward approach to compute unbiased Monte Carlo-based estimates. We show that VinePPO consistently outperforms PPO and other RL-free baselines across MATH and GSM8K datasets.
  arXiv  Detail & Related papers  (2024-10-02T15:49:30Z)
• Deep Reinforcement Learning and Mean-Variance Strategies for Responsible Portfolio Optimization [49.396692286192206]
  We study the use of deep reinforcement learning for responsible portfolio optimization by incorporating ESG states and objectives. Our results show that deep reinforcement learning policies can provide competitive performance against mean-variance approaches for responsible portfolio allocation.
  arXiv  Detail & Related papers  (2024-03-25T12:04:03Z)
• An Ensemble Method of Deep Reinforcement Learning for Automated Cryptocurrency Trading [16.78239969166596]
  We propose an ensemble method to improve the generalization performance of trading strategies trained by deep reinforcement learning algorithms. Our proposed ensemble method improves the out-of-sample performance compared with the benchmarks of a deep reinforcement learning strategy and a passive investment strategy.
  arXiv  Detail & Related papers  (2023-07-27T04:00:09Z)
• Hierarchical Deep Reinforcement Learning for VWAP Strategy Optimization [9.430129571478629]
  We propose a deep learning and hierarchical reinforcement learning architecture to capture market patterns and execute orders from different temporal scales. Our approach outperforms baselines in terms of VWAP slippage, with an average cost saving of 1.16 base points compared to the optimal baseline.
  arXiv  Detail & Related papers  (2022-12-11T07:35:26Z)
• Asset Allocation: From Markowitz to Deep Reinforcement Learning [2.0305676256390934]
  Asset allocation is an investment strategy that aims to balance risk and reward by constantly redistributing the portfolio's assets. We conduct an extensive benchmark study to determine the efficacy and reliability of a number of optimization techniques.
  arXiv  Detail & Related papers  (2022-07-14T14:44:04Z)
• Policy Gradient Bayesian Robust Optimization for Imitation Learning [49.881386773269746]
  We derive a novel policy gradient-style robust optimization approach, PG-BROIL, to balance expected performance and risk. Results suggest PG-BROIL can produce a family of behaviors ranging from risk-neutral to risk-averse.
  arXiv  Detail & Related papers  (2021-06-11T16:49:15Z)
• Universal Trading for Order Execution with Oracle Policy Distillation [99.57416828489568]
  We propose a novel universal trading policy optimization framework to bridge the gap between the noisy yet imperfect market states and the optimal action sequences for order execution. We show that our framework can better guide the learning of the common policy towards practically optimal execution by an oracle teacher with perfect information.
  arXiv  Detail & Related papers  (2021-01-28T05:52:18Z)
• Time your hedge with Deep Reinforcement Learning [0.0]
  Deep Reinforcement Learning (DRL) can tackle this challenge by creating a dynamic dependency between market information and hedging strategies allocation decisions. We present a realistic and augmented DRL framework that: (i) uses additional contextual information to decide an action, (ii) has a one period lag between observations and actions to account for one day lag turnover of common asset managers to rebalance their hedge, (iii) is fully tested in terms of stability and robustness thanks to a repetitive train test method called anchored walk forward training, similar in spirit to k fold cross validation for time series and (iv) allows managing leverage of our hedging
  arXiv  Detail & Related papers  (2020-09-16T06:43:41Z)
• Deep Learning for Portfolio Optimization [5.833272638548154]
  Instead of selecting individual assets, we trade Exchange-Traded Funds (ETFs) of market indices to form a portfolio. We compare our method with a wide range of algorithms with results showing that our model obtains the best performance over the testing period.
  arXiv  Detail & Related papers  (2020-05-27T21:28:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.