On the Convergence of Jacobian-Free Backpropagation for Optimal Control Problems with Implicit Hamiltonians

• URL: http://arxiv.org/abs/2602.00921v1
• Date: Sat, 31 Jan 2026 22:25:46 GMT
• Title: On the Convergence of Jacobian-Free Backpropagation for Optimal Control Problems with Implicit Hamiltonians
• Abstract summary: Optimal feedback control with implicit Hamiltonians poses a fundamental challenge for learning-based value function methods.<n>Recent workcitegelphman2025end introduced an implicit deep learning approach using Jacobian-Free Backpropagation (JFB) to address this setting.<n>We establish convergence guarantees for JFB in the minibatch setting, showing that the resulting updates converge to stationary points of the expected optimal control objective.
• Score: 0.8244545230770907
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Optimal feedback control with implicit Hamiltonians poses a fundamental challenge for learning-based value function methods due to the absence of closed-form optimal control laws. Recent work~\cite{gelphman2025end} introduced an implicit deep learning approach using Jacobian-Free Backpropagation (JFB) to address this setting, but only established sample-wise descent guarantees. In this paper, we establish convergence guarantees for JFB in the stochastic minibatch setting, showing that the resulting updates converge to stationary points of the expected optimal control objective. We further demonstrate scalability on substantially higher-dimensional problems, including multi-agent optimal consumption and swarm-based quadrotor and bicycle control. Together, our results provide both theoretical justification and empirical evidence for using JFB in high-dimensional optimal control with implicit Hamiltonians.

Related papers

• Language Generation as Optimal Control: Closed-Loop Diffusion in Latent Control Space [99.30250506439678]
  This work reformulates language generation as an optimal control problem.<n>It provides a unified theoretical perspective to analyze autoregressive and diffusion models.
  arXiv  Detail & Related papers  (2026-05-14T08:13:43Z)
• Too Correct to Learn: Reinforcement Learning on Saturated Reasoning Data [55.84428098924793]
  We propose Constrained Uniform Top-K Sampling (CUTS), a parameter-free decoding strategy enforcing structure-preserving exploration.<n>We integrate this into Mixed-CUTS, a training framework synergizing exploitative and exploratory rollouts to amplify intra-group advantage variance.<n> Notably, Mixed-CUTS improves Pass@1 accuracy on the challenging AIME25 benchmark by up to 15.1% over standard GRPO.
  arXiv  Detail & Related papers  (2026-04-20T16:43:28Z)
• Optimization-Free Constrained Control with Guaranteed Recursive Feasibility: A CBF-Based Reference Governor Approach [6.08496365939333]
  This letter presents a constrained control framework that integrates Explicit Reference Governors (ERG) with Control Barrier Functions (CBF)<n>We formulate the reference update as a virtual control input for an augmented system, governed by a smooth barrier function constructed from the softmin aggregation of Dynamic Safety Margins (DSMs)<n>This allows for the derivation of an explicit, closed-form reference update law that strictly enforces safety while minimizing deviation from a nominal trajectory.
  arXiv  Detail & Related papers  (2026-04-05T07:22:02Z)
• Adjoint Matching through the Lens of the Stochastic Maximum Principle in Optimal Control [12.577114890059187]
  Reward fine-tuning of diffusion and flow models and sampling from tilted or Boltzmann can both be formulated as optimal control (SOC) problems.<n>We revisit a proposed SOC-based method for learning optimal controls, and place it on a rigorous footing by deriving it from the Maximum Principle (SMP)<n>We show that iterations can be precisely interpreted as a continuous-time method of successive approximations induced by the SMP.
  arXiv  Detail & Related papers  (2026-03-28T13:01:01Z)
• Diffusion Controller: Framework, Algorithms and Parameterization [54.82539154511621]
  We present a unified control-theoretic view that casts reverse diffusion sampling as state-only control within (generalized) linearly-solvable Markov Decision Processes.<n>Under this framework, control acts by reweighting the pretrained reverse-time transition kernels, balancing terminal objectives against an $f$-divergence cost.<n>Experiments on Stable Diffusion v1.4 show consistent gains in preference-alignment win rates and improved quality-efficiency trade-offs.
  arXiv  Detail & Related papers  (2026-03-07T01:49:59Z)
• On the Limits of Test-Time Compute: Sequential Reward Filtering for Better Inference [71.09125259964684]
  Test-time compute (TTC) has become an increasingly prominent paradigm for enhancing large language models (LLMs)<n>We study reward-filtered sequential inference, a simple procedure that selectively incorporates only high-reward generations into the context.<n>On the theoretical side, we show that reward-filtered sequential inference yields strictly stronger guarantees than standard TTC paradigms.
  arXiv  Detail & Related papers  (2025-12-04T08:21:33Z)
• GegenbauerNet: Finding the Optimal Compromise in the GNN Flexibility-Stability Trade-off [0.0]
  Spectral Graph Neural Networks (GNNs) operating in the canonical [-1, 1] domain face a fundamental Flexibility-Stability Trade-off.<n>We propose textbfGegenbauerNet, a novel GNN filter based on the Gegenbauers symmetry.<n>We demonstrate that GegenbauerNet achieves superior performance in the key local filtering regime.
  arXiv  Detail & Related papers  (2025-11-04T19:39:29Z)
• End-to-End Training of High-Dimensional Optimal Control with Implicit Hamiltonians via Jacobian-Free Backpropagation [0.8244545230770907]
  We propose an end-to-end implicit deep learning approach that directly parameterizes the value function to learn optimal control laws.<n>We show that our approach effectively learns high-dimensional feedback controllers across multiple scenarios involving implicit Hamiltonians.
  arXiv  Detail & Related papers  (2025-10-01T00:03:08Z)
• Continuous Policy and Value Iteration for Stochastic Control Problems and Its Convergence [8.65436459753278]
  We introduce a continuous policy iteration algorithm where the approximations of the value function of a control problem and the optimal control are simultaneously updated through Langevin-type dynamics.
  arXiv  Detail & Related papers  (2025-06-09T18:20:21Z)
• Supervised Optimism Correction: Be Confident When LLMs Are Sure [91.7459076316849]
  We establish a novel theoretical connection between supervised fine-tuning and offline reinforcement learning.<n>We show that the widely used beam search method suffers from unacceptable over-optimism.<n>We propose Supervised Optimism Correction, which introduces a simple yet effective auxiliary loss for token-level $Q$-value estimations.
  arXiv  Detail & Related papers  (2025-04-10T07:50:03Z)
• You KAN Do It in a Single Shot: Plug-and-Play Methods with Single-Instance Priors [10.726369475010818]
  We introduce KAN-Play, an optimisation framework that incorporates Kologorov- Networks (KANs) as denoisers.<n>KAN-Play is specifically designed to solve problems with single-instance inverse priors, where only a single noisy observation is available.
  arXiv  Detail & Related papers  (2024-12-09T04:55:18Z)
• Stochastic Optimal Control Matching [53.156277491861985]
  Our work introduces Optimal Control Matching (SOCM), a novel Iterative Diffusion Optimization (IDO) technique for optimal control. The control is learned via a least squares problem by trying to fit a matching vector field. Experimentally, our algorithm achieves lower error than all the existing IDO techniques for optimal control.
  arXiv  Detail & Related papers  (2023-12-04T16:49:43Z)
• Optimal control for state preparation in two-qubit open quantum systems driven by coherent and incoherent controls via GRAPE approach [77.34726150561087]
  We consider a model of two qubits driven by coherent and incoherent time-dependent controls. The dynamics of the system is governed by a Gorini-Kossakowski-Sudarshan-Lindblad master equation. We study evolution of the von Neumann entropy, purity, and one-qubit reduced density matrices under optimized controls.
  arXiv  Detail & Related papers  (2022-11-04T15:20:18Z)
• Error-based Knockoffs Inference for Controlled Feature Selection [49.99321384855201]
  We propose an error-based knockoff inference method by integrating the knockoff features, the error-based feature importance statistics, and the stepdown procedure together. The proposed inference procedure does not require specifying a regression model and can handle feature selection with theoretical guarantees.
  arXiv  Detail & Related papers  (2022-03-09T01:55:59Z)
• Gaussian Process-based Min-norm Stabilizing Controller for Control-Affine Systems with Uncertain Input Effects and Dynamics [90.81186513537777]
  We propose a novel compound kernel that captures the control-affine nature of the problem. We show that this resulting optimization problem is convex, and we call it Gaussian Process-based Control Lyapunov Function Second-Order Cone Program (GP-CLF-SOCP)
  arXiv  Detail & Related papers  (2020-11-14T01:27:32Z)
• Adaptive Control and Regret Minimization in Linear Quadratic Gaussian (LQG) Setting [91.43582419264763]
  We propose LqgOpt, a novel reinforcement learning algorithm based on the principle of optimism in the face of uncertainty. LqgOpt efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model.
  arXiv  Detail & Related papers  (2020-03-12T19:56:38Z)
• Convergence Guarantees of Policy Optimization Methods for Markovian Jump Linear Systems [3.3343656101775365]
  We show that the Gauss-Newton method converges to the optimal state feedback controller for MJLS at a linear rate if at a controller which stabilizes the closed-loop dynamics in the mean sense. We present an example to support our theory.
  arXiv  Detail & Related papers  (2020-02-10T21:13:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.