Exploratory Mean-Variance Portfolio Optimization with Regime-Switching Market Dynamics

• URL: http://arxiv.org/abs/2501.16659v1
• Date: Tue, 28 Jan 2025 02:48:41 GMT
• Title: Exploratory Mean-Variance Portfolio Optimization with Regime-Switching Market Dynamics
• Authors: Yuling Max Chen, Bin Li, David Saunders,
• Abstract summary: We study a regime-switching market setting and apply reinforcement learning techniques to assist informed exploration within the control space. In a real market data study, EMVRS with OC learning outperforms its counterparts with the highest mean and reasonably low volatility of the annualized portfolio returns.
• Score: 3.6149777601911097
• License:
• Abstract: Considering the continuous-time Mean-Variance (MV) portfolio optimization problem, we study a regime-switching market setting and apply reinforcement learning (RL) techniques to assist informed exploration within the control space. We introduce and solve the Exploratory Mean Variance with Regime Switching (EMVRS) problem. We also present a Policy Improvement Theorem. Further, we recognize that the widely applied Temporal Difference (TD) learning is not adequate for the EMVRS context, hence we consider Orthogonality Condition (OC) learning, leveraging the martingale property of the induced optimal value function from the analytical solution to EMVRS. We design a RL algorithm that has more meaningful parameterization using the market parameters and propose an updating scheme for each parameter. Our empirical results demonstrate the superiority of OC learning over TD learning with a clear convergence of the market parameters towards their corresponding ``grounding true" values in a simulated market scenario. In a real market data study, EMVRS with OC learning outperforms its counterparts with the highest mean and reasonably low volatility of the annualized portfolio returns.

Related papers

• Reinforcement-Learning Portfolio Allocation with Dynamic Embedding of Market Information [12.032301674764552]
  We develop a portfolio allocation framework that leverages deep learning techniques to address challenges arising from high-dimensional, non-stationary, and low-signal-to-noise market information. We design a reinforcement learning framework that integrates generative autoencoders and online meta-learning to dynamically embed market information. Empirical analysis based on the top 500 U.S. stocks demonstrates that our framework outperforms common portfolio benchmarks.
  arXiv  Detail & Related papers  (2025-01-29T20:56:59Z)
• MetaTrading: An Immersion-Aware Model Trading Framework for Vehicular Metaverse Services [94.61039892220037]
  We present a novel immersion-aware model trading framework that incentivizes metaverse users (MUs) to contribute learning models for augmented reality (AR) services in the vehicular metaverse. Considering dynamic network conditions and privacy concerns, we formulate the reward decisions of MSPs as a multi-agent Markov decision process. Experimental results demonstrate that the proposed framework can effectively provide higher-value models for object detection and classification in AR services on real AR-related vehicle datasets.
  arXiv  Detail & Related papers  (2024-10-25T16:20:46Z)
• EdgeRL: Reinforcement Learning-driven Deep Learning Model Inference Optimization at Edge [2.8946323553477704]
  We propose EdgeRL framework that seeks to strike balance by using an Advantage Actor-Critic (A2C) Reinforcement Learning (RL) approach. We evaluate the benefits of EdgeRL framework in terms of end device energy savings, inference accuracy improvement, and end-to-end inference latency reduction.
  arXiv  Detail & Related papers  (2024-10-16T04:31:39Z)
• What Teaches Robots to Walk, Teaches Them to Trade too -- Regime Adaptive Execution using Informed Data and LLMs [1.6317061277457001]
  We introduce an innovative approach that leverages world knowledge of pretrained LLMs (aka. 'privileged information' in robotics) and dynamically adapts them using intrinsic, natural market rewards. Strong empirical results demonstrate the efficacy of our method in adapting to regime shifts in financial markets. The proposed algorithmic framework outperforms best-performing SOTA LLM models on the existing (FLARE) benchmark stock-movement (SM) tasks by more than 15% improved accuracy.
  arXiv  Detail & Related papers  (2024-06-20T00:17:28Z)
• Let's reward step by step: Step-Level reward model as the Navigators for Reasoning [64.27898739929734]
  Process-Supervised Reward Model (PRM) furnishes LLMs with step-by-step feedback during the training phase. We propose a greedy search algorithm that employs the step-level feedback from PRM to optimize the reasoning pathways explored by LLMs. To explore the versatility of our approach, we develop a novel method to automatically generate step-level reward dataset for coding tasks and observed similar improved performance in the code generation tasks.
  arXiv  Detail & Related papers  (2023-10-16T05:21:50Z)
• Commodities Trading through Deep Policy Gradient Methods [0.0]
  It formulates the commodities trading problem as a continuous, discrete-time dynamical system. Two policy algorithms, namely actor-based and actor-critic-based approaches, are introduced. Backtesting on front-month natural gas futures demonstrates that DRL models increase the Sharpe ratio by $83%$ compared to the buy-and-hold baseline.
  arXiv  Detail & Related papers  (2023-08-10T17:21:12Z)
• Generalized Parametric Contrastive Learning [60.62901294843829]
  Generalized Parametric Contrastive Learning (GPaCo/PaCo) works well on both imbalanced and balanced data. Experiments on long-tailed benchmarks manifest the new state-of-the-art for long-tailed recognition.
  arXiv  Detail & Related papers  (2022-09-26T03:49:28Z)
• Efficient Model-based Multi-agent Reinforcement Learning via Optimistic Equilibrium Computation [93.52573037053449]
  H-MARL (Hallucinated Multi-Agent Reinforcement Learning) learns successful equilibrium policies after a few interactions with the environment. We demonstrate our approach experimentally on an autonomous driving simulation benchmark.
  arXiv  Detail & Related papers  (2022-03-14T17:24:03Z)
• ARISE: ApeRIodic SEmi-parametric Process for Efficient Markets without Periodogram and Gaussianity Assumptions [91.3755431537592]
  We present the ApeRI-miodic (ARISE) process for investigating efficient markets. The ARISE process is formulated as an infinite-sum of some known processes and employs the aperiodic spectrum estimation. In practice, we apply the ARISE function to identify the efficiency of real-world markets.
  arXiv  Detail & Related papers  (2021-11-08T03:36:06Z)
• Reinforced Deep Markov Models With Applications in Automatic Trading [0.0]
  We propose a model-based RL approach, coined Reinforced Deep Markov Model (RDMM) RDMM integrates desirable properties of a reinforcement learning algorithm acting as an automatic trading system. Tests show that the RDMM is data-efficient and provides financial gains compared to the benchmarks in the optimal execution problem.
  arXiv  Detail & Related papers  (2020-11-09T12:46:30Z)
• Adaptive Control and Regret Minimization in Linear Quadratic Gaussian (LQG) Setting [91.43582419264763]
  We propose LqgOpt, a novel reinforcement learning algorithm based on the principle of optimism in the face of uncertainty. LqgOpt efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model.
  arXiv  Detail & Related papers  (2020-03-12T19:56:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.