Deep reinforcement learning for optimal trading with partial information

• URL: http://arxiv.org/abs/2511.00190v1
• Date: Fri, 31 Oct 2025 18:48:59 GMT
• Title: Deep reinforcement learning for optimal trading with partial information
• Authors: Andrea Macrì, Sebastian Jaimungal, Fabrizio Lillo,
• Abstract summary: We study an optimal trading problem, where a trading signal follows an Ornstein-Uhlenbeck process with regime-switching dynamics.<n>We employ a blend of RL and Recurrent Neural Networks (RNN) in order to make the most at extracting underlying information from the trading signal with latent parameters.
• Score: 0.254890465057467
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reinforcement Learning (RL) applied to financial problems has been the subject of a lively area of research. The use of RL for optimal trading strategies that exploit latent information in the market is, to the best of our knowledge, not widely tackled. In this paper we study an optimal trading problem, where a trading signal follows an Ornstein-Uhlenbeck process with regime-switching dynamics. We employ a blend of RL and Recurrent Neural Networks (RNN) in order to make the most at extracting underlying information from the trading signal with latent parameters. The latent parameters driving mean reversion, speed, and volatility are filtered from observations of the signal, and trading strategies are derived via RL. To address this problem, we propose three Deep Deterministic Policy Gradient (DDPG)-based algorithms that integrate Gated Recurrent Unit (GRU) networks to capture temporal dependencies in the signal. The first, a one -step approach (hid-DDPG), directly encodes hidden states from the GRU into the RL trader. The second and third are two-step methods: one (prob-DDPG) makes use of posterior regime probability estimates, while the other (reg-DDPG) relies on forecasts of the next signal value. Through extensive simulations with increasingly complex Markovian regime dynamics for the trading signal's parameters, as well as an empirical application to equity pair trading, we find that prob-DDPG achieves superior cumulative rewards and exhibits more interpretable strategies. By contrast, reg-DDPG provides limited benefits, while hid-DDPG offers intermediate performance with less interpretable strategies. Our results show that the quality and structure of the information supplied to the agent are crucial: embedding probabilistic insights into latent regimes substantially improves both profitability and robustness of reinforcement learning-based trading strategies.

Related papers

• Towards Sample-Efficient and Stable Reinforcement Learning for LLM-based Recommendation [56.92367609590823]
  Long Chain-of-Thought (Long CoT) reasoning has shown promise in Large Language Models (LLMs)<n>We argue that Long CoT is inherently ill-suited for the sequential recommendation domain.<n>We propose RISER, a novel Reinforced Item Space Exploration framework for Recommendation.
  arXiv  Detail & Related papers  (2026-01-31T10:02:43Z)
• Consolidation or Adaptation? PRISM: Disentangling SFT and RL Data via Gradient Concentration [56.074760766965085]
  PRISM achieves a dynamics-aware framework that arbitrates data based on its degree of cognitive conflict with the model's existing knowledge.<n>Our findings suggest that disentangling data based on internal optimization regimes is crucial for scalable and robust agent alignment.
  arXiv  Detail & Related papers  (2026-01-12T05:43:20Z)
• Deep Reinforcement Learning for Automated Stock Trading: An Ensemble Strategy [10.667441394970071]
  We propose an ensemble strategy that employs deep reinforcement schemes to learn a stock trading strategy by maximizing investment return.<n>We train a deep reinforcement learning agent and obtain an ensemble trading strategy using three actor-critic based algorithms.<n>The proposed deep ensemble strategy is shown to outperform the three individual algorithms and two baselines in terms of the risk-adjusted return measured by the Sharpe ratio.
  arXiv  Detail & Related papers  (2025-11-15T09:15:10Z)
• Demystifying Reinforcement Learning in Agentic Reasoning [90.3737088727791]
  We conduct a comprehensive and systematic investigation to demystify reinforcement learning in agentic reasoning.<n>We highlight our key insights: (i) replacing stitched synthetic trajectories with real end-to-end tool-use trajectories yields a far stronger SFT.<n> Exploration-friendly techniques are crucial for agentic RL, such as clip higher, overlong reward shaping, and maintaining adequate policy entropy could improve the training efficiency.
  arXiv  Detail & Related papers  (2025-10-13T17:57:15Z)
• Token Hidden Reward: Steering Exploration-Exploitation in Group Relative Deep Reinforcement Learning [64.04741347596938]
  We introduce Token Hidden Reward (THR), a token-level metric that quantifies each token's influence on the likelihood of correct responses.<n>We find that training dynamics are dominated by a small subset of tokens with high absolute THR values.<n>This insight suggests a THR-guided reweighting algorithm that modulates GRPO's learning signals to explicitly bias training toward exploitation or exploration.
  arXiv  Detail & Related papers  (2025-10-04T04:49:44Z)
• Group-Relative REINFORCE Is Secretly an Off-Policy Algorithm: Demystifying Some Myths About GRPO and Its Friends [64.71326476563213]
  Off-policy reinforcement learning for large language models (LLMs) is attracting growing interest.<n>We present a first-principles derivation for grouprelative REINFORCE without assuming a specific training data distribution.<n>This perspective yields two general principles for adapting REINFORCE to off-policy settings.
  arXiv  Detail & Related papers  (2025-09-29T02:34:54Z)
• Learning Efficient and Generalizable Graph Retriever for Knowledge-Graph Question Answering [75.12322966980003]
  Large Language Models (LLMs) have shown strong inductive reasoning ability across various domains.<n>Most existing RAG pipelines rely on unstructured text, limiting interpretability and structured reasoning.<n>Recent studies have explored integrating knowledge graphs with LLMs for knowledge graph question answering.<n>We propose RAPL, a novel framework for efficient and effective graph retrieval in KGQA.
  arXiv  Detail & Related papers  (2025-06-11T12:03:52Z)
• Your Offline Policy is Not Trustworthy: Bilevel Reinforcement Learning for Sequential Portfolio Optimization [82.03139922490796]
  Reinforcement learning (RL) has shown significant promise for sequential portfolio optimization tasks, such as stock trading, where the objective is to maximize cumulative returns while minimizing risks using historical data.<n>Traditional RL approaches often produce policies that merely memorize the optimal yet impractical buying and selling behaviors within the fixed dataset.<n>Our approach frames portfolio optimization as a new type of partial-offline RL problem and makes two technical contributions.
  arXiv  Detail & Related papers  (2025-05-19T06:37:25Z)
• Risk-averse policies for natural gas futures trading using distributional reinforcement learning [0.0]
  This paper studies the effectiveness of three distributional RL algorithms for natural gas futures trading.<n>To the best of our knowledge, these algorithms have never been applied in a trading context.<n>We show that training C51 and IQN to maximize CVaR produces risk-sensitive policies with adjustable risk aversion.
  arXiv  Detail & Related papers  (2025-01-08T11:11:25Z)
• Hierarchical Reinforced Trader (HRT): A Bi-Level Approach for Optimizing Stock Selection and Execution [0.9553307596675155]
  We introduce the Hierarchical Reinforced Trader (HRT), a novel trading strategy employing a bi-level Hierarchical Reinforcement Learning framework. HRT integrates a Proximal Policy Optimization (PPO)-based High-Level Controller (HLC) for strategic stock selection with a Deep Deterministic Policy Gradient (DDPG)-based Low-Level Controller (LLC) tasked with optimizing trade executions to enhance portfolio value.
  arXiv  Detail & Related papers  (2024-10-19T01:29:38Z)
• Logic-Q: Improving Deep Reinforcement Learning-based Quantitative Trading via Program Sketch-based Tuning [9.039809980024852]
  We propose a universal logic-guided deep reinforcement learning framework for Q-trading, called Logic-Q.<n>In particular, Logic-Q adopts the program synthesis by sketching paradigm and introduces a logic-guided model design that leverages a lightweight, plug-and-play market trend-aware program sketch to determine the market trend.<n>Extensive evaluations of two popular quantitative trading tasks demonstrate that Logic-Q can significantly improve the performance of previous state-of-the-art DRL trading strategies.
  arXiv  Detail & Related papers  (2023-10-09T09:20:13Z)
• Bayesian Distributional Policy Gradients [2.28438857884398]
  Distributional Reinforcement Learning maintains the entire probability distribution of the reward-to-go, i.e. the return. Bayesian Distributional Policy Gradients (BDPG) uses adversarial training in joint-contrastive learning to estimate a variational posterior from the returns.
  arXiv  Detail & Related papers  (2021-03-20T23:42:50Z)
• An Application of Deep Reinforcement Learning to Algorithmic Trading [4.523089386111081]
  This scientific research paper presents an innovative approach based on deep reinforcement learning (DRL) to solve the algorithmic trading problem. It proposes a novel DRL trading strategy so as to maximise the resulting Sharpe ratio performance indicator on a broad range of stock markets. The training of the resulting reinforcement learning (RL) agent is entirely based on the generation of artificial trajectories from a limited set of stock market historical data.
  arXiv  Detail & Related papers  (2020-04-07T14:57:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.