Robust $Q$-learning for mean-field control under Wasserstein uncertainty in common noise

• URL: http://arxiv.org/abs/2606.20356v1
• Date: Thu, 18 Jun 2026 15:20:00 GMT
• Title: Robust $Q$-learning for mean-field control under Wasserstein uncertainty in common noise
• Abstract summary: We present a robust $Q$-learning algorithm for discrete-time mean-field control problems under Wasserstein uncertainty in the common noise law.<n>We establish its convergence together with finite-time iteration bounds for both synchronous and asynchronous learning schemes.
• Score: 5.5004369279667396
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this article, we present a robust $Q$-learning algorithm for discrete-time mean-field control problems under Wasserstein uncertainty in the common noise law. The algorithm combines a quantization-and-projection scheme with a Wasserstein dual reformulation on the common-noise space. We establish its convergence together with finite-time iteration bounds for both synchronous and asynchronous learning schemes. Numerical experiments on systemic risk and epidemic models compare the asynchronous implementation with an idealized Bellman iteration, illustrate the robustness-performance tradeoff under common-noise misspecification, and report the observed convergence behavior of the asynchronous $Q$-learning algorithm.

Related papers

• Scalable Lindblad Noise Learning via Stochastic Tensor-Network Simulation [48.887627688666974]
  Learning dissipation rates in large-scale open quantum systems is a major obstacle for near-term quantum technologies.<n>Here, we propose a scalable noise-learning framework for Lindblad dissipation rates that combines a simulation method, the Jump Method (TJM)<n>We demonstrate the approach on two noise models in the Ising model, in which independent dissipation rates are learned for each site up to $N_mathrmsite=16$.<n>This combination of scalable numerics and rigorous theoretical guarantees positions TJM-based noise learning as a practical foundation for characterizing dissipation in large quantum devices.
  arXiv  Detail & Related papers  (2026-08-25T15:07:11Z)
• PedestrianDiffusion: Multimodal Generative Denoising and Dense State Estimation for Inertial Navigation [1.8620637029128544]
  PedestrianDiffusion is a generative framework that reformulates dense 6D state estimation as a continuous conditional denoising process.<n>It achieves state-of-the-art performance, exhibiting unprecedented robustness to impulse perturbations and coupled 6D kinematic drift.<n>This work provides a rigorous algorithmic blueprint for next-generation Neural Inertial Measurement Units (N-IMUs)
  arXiv  Detail & Related papers  (2026-07-03T14:04:30Z)
• Provable Mixed-Noise Learning with Flow-Matching [6.687571760931077]
  We study inverse problems with mixed noise, modeled as a combination of additive and multiplicative Gaussian components.<n>Motivated by recent advances in flow-based generative modeling, we propose a novel inference framework based on conditional flow matching embedded within an Expectation-Maximization (EM) algorithm.
  arXiv  Detail & Related papers  (2025-08-25T15:30:12Z)
• Impact of Temporally Correlated Dephasing Noise on the Fidelity of the 2-Qubit Deutsch-Jozsa Algorithm [0.76146285961466]
  Noise in quantum systems often exhibits temporal correlations, leading to non-Markovian dynamics.<n>This paper investigates the impact of temporally correlated noise on the fidelity of the 2-qubit Deutsch-Jozsa algorithm.
  arXiv  Detail & Related papers  (2025-06-05T18:48:50Z)
• Stochastic action for the entanglement of a noisy monitored two-qubit system [55.2480439325792]
  We study the effect of local unitary noise on the entanglement evolution of a two-qubit system subject to local monitoring and inter-qubit coupling. We construct a Hamiltonian by incorporating the noise into the Chantasri-Dressel-Jordan path integral and use it to identify the optimal entanglement dynamics. Numerical investigation of long-time steady-state entanglement reveals a non-monotonic relationship between concurrence and noise strength.
  arXiv  Detail & Related papers  (2024-03-13T11:14:10Z)
• Cooperative Learning with Gaussian Processes for Euler-Lagrange Systems Tracking Control under Switching Topologies [9.838373797093245]
  This work presents an innovative learning-based approach to tackle the tracking control problem of Euler-Lagrange multi-agent systems. A standout feature is its exceptional efficiency in deriving the aggregation weights achieved. Simulation experiments validate the protocol's efficacy in effectively managing complex scenarios.
  arXiv  Detail & Related papers  (2024-02-05T14:33:52Z)
• Provable Guarantees for Generative Behavior Cloning: Bridging Low-Level Stability and High-Level Behavior [51.60683890503293]
  We propose a theoretical framework for studying behavior cloning of complex expert demonstrations using generative modeling. We show that pure supervised cloning can generate trajectories matching the per-time step distribution of arbitrary expert trajectories.
  arXiv  Detail & Related papers  (2023-07-27T04:27:26Z)
• A Deep Unrolling Model with Hybrid Optimization Structure for Hyperspectral Image Deconvolution [50.13564338607482]
  We propose a novel optimization framework for the hyperspectral deconvolution problem, called DeepMix.<n>It consists of three distinct modules, namely, a data consistency module, a module that enforces the effect of the handcrafted regularizers, and a denoising module.<n>This work proposes a context aware denoising module designed to sustain the advancements achieved by the cooperative efforts of the other modules.
  arXiv  Detail & Related papers  (2023-06-10T08:25:16Z)
• PAPAL: A Provable PArticle-based Primal-Dual ALgorithm for Mixed Nash Equilibrium [58.26573117273626]
  We consider the non-AL equilibrium nonconptotic objective function in two-player zero-sum continuous games. Our novel insights into the particle-based algorithms for continuous distribution strategies are presented.
  arXiv  Detail & Related papers  (2023-03-02T05:08:15Z)
• Distributed Bayesian Learning of Dynamic States [65.7870637855531]
  The proposed algorithm is a distributed Bayesian filtering task for finite-state hidden Markov models. It can be used for sequential state estimation, as well as for modeling opinion formation over social networks under dynamic environments.
  arXiv  Detail & Related papers  (2022-12-05T19:40:17Z)
• Distributional Hamilton-Jacobi-Bellman Equations for Continuous-Time Reinforcement Learning [39.07307690074323]
  We consider the problem of predicting the distribution of returns obtained by an agent interacting in a continuous-time environment. Accurate return predictions have proven useful for determining optimal policies for risk-sensitive control, state representations, multiagent coordination, and more. We propose a tractable algorithm for approximately solving the distributional HJB based on a JKO scheme, which can be implemented in an online control algorithm.
  arXiv  Detail & Related papers  (2022-05-24T16:33:54Z)
• Robust Reinforcement Learning with Wasserstein Constraint [49.86490922809473]
  We show the existence of optimal robust policies, provide a sensitivity analysis for the perturbations, and then design a novel robust learning algorithm. The effectiveness of the proposed algorithm is verified in the Cart-Pole environment.
  arXiv  Detail & Related papers  (2020-06-01T13:48:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.