Continuous-Time Reinforcement Learning for Controlled Hawkes Jump-Diffusions

• URL: http://arxiv.org/abs/2608.19151v1
• Date: Wed, 19 Aug 2026 17:40:23 GMT
• Title: Continuous-Time Reinforcement Learning for Controlled Hawkes Jump-Diffusions
• Abstract summary: We study control of multivariate Hawkes-driven differential equations with machine learning algorithms in a non-Markovian setting.<n>We first develop a finite-dimensional Markovianization procedure and algorithm to approximate Hawkes processes with mixtures of exponential kernels.<n>We then formulate continuous-time deterministic policy gradient learning on the Markovianized approximation of the problem, called Hawkes-CT DDPG.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study stochastic control of multivariate Hawkes-driven stochastic differential equations with machine learning algorithms in a non-Markovian setting. Due to the path dependence of the memory of the Hawkes intensity, this problem does not fall within classical stochastic control theory outside particular Markovian kernels. We first develop a finite-dimensional Markovianization procedure and algorithm to approximate multivariate Hawkes processes with mixtures of exponential kernels. We prove the convergence of the Markovianized approximation of the Hawkes process, its intensity, and the value of the problem to the original non-Markovian processes and the value of the primal problem. We then formulate continuous-time deterministic policy gradient learning on the Markovianized approximation of the problem, called Hawkes-CT DDPG. We propose a model-free algorithm to solve the non-Markovian Hawkes-driven optimization by observing only the event times of the process, the realization of the solution to the SDE, and a chosen set of decay filters, while the Hawkes kernel coefficients remain unknown. We compare our continuous time reinforcement learning Hawkes-CT DDPG method with discrete time reinforcement learning techniques under three different types of kernels: simple exponential, Erlang, and power-law kernels.

Related papers

• Learning Ergodic Dynamical Systems from a Finite Trajectory [10.59816347247158]
  We study discrete-time autonomous systems defining time-homogeneous Markov processes.<n>We first focus on estimating the optimal one-step prediction function by nonlinear least squares.<n>We extend the framework to higher-order systems and finite-state spaces.
  arXiv  Detail & Related papers  (2026-07-24T15:21:51Z)
• Regret and Sample Complexity of Online Q-Learning via Concentration of Stochastic Approximation with Time-Inhomogeneous Markov Chains [23.565936864449636]
  We present the first high-probability regret bound for classical online Q-learning in infinite-horizon discounted Markov decision processes.<n>For sufficiently large gaps, the regret is sublinear, while for small gaps it deteriorates and can approach linear growth.
  arXiv  Detail & Related papers  (2026-02-18T08:47:07Z)
• Better LMO-based Momentum Methods with Second-Order Information [48.580700968416444]
  Hessian-Corrected Momentum (HCM) aims to improve momentum convergence rates.<n>Hessian-Corrected Momentum can adapt to the geometry of the problem and achieve a faster rate than traditional momentum.<n>We extend the Linear Minimization Oracle framework by integrating HCM, and provide convergence guarantees under relaxed smoothness and arbitrary norm settings.
  arXiv  Detail & Related papers  (2025-12-15T11:43:09Z)
• Quartic quantum speedups for community detection [84.14713515477784]
  We develop a quantum algorithm for hypergraph community detection that achieves a quartic quantum speedup.<n>Our algorithm is based on the Kikuchi method, which we extend beyond previously considered problems such as PCA and $p$XORSAT.
  arXiv  Detail & Related papers  (2025-10-09T17:35:17Z)
• Variational Quantum Optimization with Continuous Bandits [1.6686882054452727]
  We introduce a novel approach to variational Quantum algorithms (VQA) via continuous bandits.<n>VQA are a class of hybrid Quantum-classical algorithms where the parameters of Quantum circuits are optimized by classical algorithms.
  arXiv  Detail & Related papers  (2025-02-06T12:24:30Z)
• The ODE Method for Stochastic Approximation and Reinforcement Learning with Markovian Noise [23.71604056844816]
  One fundamental challenge in analyzing an approximation algorithm is to establish its stability.<n>We extend the celebrated Borkar-Meyn theorem for stability bounded from the Martingale difference noise setting to the Markovian noise setting.
  arXiv  Detail & Related papers  (2024-01-15T17:20:17Z)
• Stochastic-Constrained Stochastic Optimization with Markovian Data [2.1756081703276]
  We study the setting where data samples are drawn from a Markov chain and thus are not independent and identically distributed. We propose two variants of drift-plus-penalty; one is for the case when the mixing time of the underlying Markov chain is known. Our algorithms apply to a more general setting of constrained online convex optimization where the sequence of constraint functions follows a Markov chain.
  arXiv  Detail & Related papers  (2023-12-07T14:09:27Z)
• First Order Methods with Markovian Noise: from Acceleration to Variational Inequalities [91.46841922915418]
  We present a unified approach for the theoretical analysis of first-order variation methods. Our approach covers both non-linear gradient and strongly Monte Carlo problems. We provide bounds that match the oracle strongly in the case of convex method optimization problems.
  arXiv  Detail & Related papers  (2023-05-25T11:11:31Z)
• Learning to Optimize with Stochastic Dominance Constraints [103.26714928625582]
  In this paper, we develop a simple yet efficient approach for the problem of comparing uncertain quantities. We recast inner optimization in the Lagrangian as a learning problem for surrogate approximation, which bypasses apparent intractability. The proposed light-SD demonstrates superior performance on several representative problems ranging from finance to supply chain management.
  arXiv  Detail & Related papers  (2022-11-14T21:54:31Z)
• A Dynamical System View of Langevin-Based Non-Convex Sampling [44.002384711340966]
  Non- sampling is a key challenge in machine learning, central to non-rate optimization in deep learning as well as to approximate its significance. Existing guarantees typically only hold for the averaged distances rather than the more desirable last-rate iterates. We develop a new framework that lifts the above issues by harnessing several tools from the theory systems.
  arXiv  Detail & Related papers  (2022-10-25T09:43:36Z)
• ARISE: ApeRIodic SEmi-parametric Process for Efficient Markets without Periodogram and Gaussianity Assumptions [91.3755431537592]
  We present the ApeRI-miodic (ARISE) process for investigating efficient markets. The ARISE process is formulated as an infinite-sum of some known processes and employs the aperiodic spectrum estimation. In practice, we apply the ARISE function to identify the efficiency of real-world markets.
  arXiv  Detail & Related papers  (2021-11-08T03:36:06Z)
• Finite-Time Convergence Rates of Nonlinear Two-Time-Scale Stochastic Approximation under Markovian Noise [2.0305676256390934]
  We study the so-called two-time-scale approximation, a simulation-based approach for finding the roots of two coupled nonlinear operators. In particular, we consider the scenario where the data in the method are generated by Markov processes, therefore, they are dependent. Under some fairly standard assumptions on the operators and the Markov processes, we provide a formula that characterizes the convergence rate of the mean square errors generated by the method to zero.
  arXiv  Detail & Related papers  (2021-04-04T15:19:19Z)
• Momentum Q-learning with Finite-Sample Convergence Guarantee [49.38471009162477]
  This paper analyzes a class of momentum-based Q-learning algorithms with finite-sample guarantee. We establish the convergence guarantee for MomentumQ with linear function approximations and Markovian sampling. We demonstrate through various experiments that the proposed MomentumQ outperforms other momentum-based Q-learning algorithms.
  arXiv  Detail & Related papers  (2020-07-30T12:27:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.