Trust Region Constrained Measure Transport in Path Space for Stochastic Optimal Control and Inference

• URL: http://arxiv.org/abs/2508.12511v1
• Date: Sun, 17 Aug 2025 22:10:35 GMT
• Title: Trust Region Constrained Measure Transport in Path Space for Stochastic Optimal Control and Inference
• Authors: Denis Blessing, Julius Berner, Lorenz Richter, Carles Domingo-Enrich, Yuanqi Du, Arash Vahdat, Gerhard Neumann,
• Abstract summary: We show that a trust region based strategy can be understood as a geometric annealing from the prior to the target measure.<n>We demonstrate in multiple optimal control applications that our novel method can improve performance significantly.
• Score: 49.11857020431547
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Solving stochastic optimal control problems with quadratic control costs can be viewed as approximating a target path space measure, e.g. via gradient-based optimization. In practice, however, this optimization is challenging in particular if the target measure differs substantially from the prior. In this work, we therefore approach the problem by iteratively solving constrained problems incorporating trust regions that aim for approaching the target measure gradually in a systematic way. It turns out that this trust region based strategy can be understood as a geometric annealing from the prior to the target measure, where, however, the incorporated trust regions lead to a principled and educated way of choosing the time steps in the annealing path. We demonstrate in multiple optimal control applications that our novel method can improve performance significantly, including tasks in diffusion-based sampling, transition path sampling, and fine-tuning of diffusion models.

Related papers

• Flow Density Control: Generative Optimization Beyond Entropy-Regularized Fine-Tuning [59.11663802446183]
  Flow and diffusion generative models can be adapted to optimize task-specific objectives while preserving prior information.<n>We introduce Flow Density Control (FDC), a simple algorithm that reduces this complex problem to a specific sequence of simpler fine-tuning tasks.<n>We derive convergence guarantees for the proposed scheme under realistic assumptions by leveraging recent understanding of mirror flows.
  arXiv  Detail & Related papers  (2025-11-27T17:19:01Z)
• Aligning Latent Spaces with Flow Priors [72.24305287508474]
  This paper presents a novel framework for aligning learnable latent spaces to arbitrary target distributions by leveraging flow-based generative models as priors.<n> Notably, the proposed method eliminates computationally expensive likelihood evaluations and avoids ODE solving during optimization.
  arXiv  Detail & Related papers  (2025-06-05T16:59:53Z)
• Optimistic Algorithms for Adaptive Estimation of the Average Treatment Effect [36.25361703897723]
  Recent advances in martingale theory have paved the way for adaptive methods that can enhance the power of downstream inference.<n>We study adaptive sampling procedures that take advantage of optimalally optimal causal inference procedures.<n>Our findings mark a step forward in advancing adaptive causal inference methods in theory and practice.
  arXiv  Detail & Related papers  (2025-02-07T05:39:32Z)
• An Efficient On-Policy Deep Learning Framework for Stochastic Optimal Control [14.832859803172846]
  We present a novel on-policy algorithm for solving optimal control (SOC) problems.<n>By leveraging the Girsanov theorem, our method directly computes on-policy gradients of the SOC objective without expensive backpropagation through differential equations or adjoint problem solutions.<n> Experimental results demonstrate substantial improvements in both computational speed and memory efficiency compared to existing approaches.
  arXiv  Detail & Related papers  (2024-10-07T16:16:53Z)
• Trust-Region Sequential Quadratic Programming for Stochastic Optimization with Random Models [57.52124921268249]
  We propose a Trust Sequential Quadratic Programming method to find both first and second-order stationary points. To converge to first-order stationary points, our method computes a gradient step in each iteration defined by minimizing a approximation of the objective subject. To converge to second-order stationary points, our method additionally computes an eigen step to explore the negative curvature the reduced Hessian matrix.
  arXiv  Detail & Related papers  (2024-09-24T04:39:47Z)
• Fully Stochastic Trust-Region Sequential Quadratic Programming for Equality-Constrained Optimization Problems [62.83783246648714]
  We propose a sequential quadratic programming algorithm (TR-StoSQP) to solve nonlinear optimization problems with objectives and deterministic equality constraints. The algorithm adaptively selects the trust-region radius and, compared to the existing line-search StoSQP schemes, allows us to utilize indefinite Hessian matrices.
  arXiv  Detail & Related papers  (2022-11-29T05:52:17Z)
• Probabilistic Control and Majorization of Optimal Control [3.2634122554914002]
  Probabilistic control design is founded on the principle that a rational agent attempts to match modelled with an arbitrary desired closed-loop system trajectory density. In this work we introduce an alternative parametrization of desired closed-loop behaviour and explore alternative proximity measures between densities.
  arXiv  Detail & Related papers  (2022-05-06T15:04:12Z)
• Outlier-Robust Sparse Estimation via Non-Convex Optimization [73.18654719887205]
  We explore the connection between high-dimensional statistics and non-robust optimization in the presence of sparsity constraints. We develop novel and simple optimization formulations for these problems. As a corollary, we obtain that any first-order method that efficiently converges to station yields an efficient algorithm for these tasks.
  arXiv  Detail & Related papers  (2021-09-23T17:38:24Z)
• Stochastic Control through Approximate Bayesian Input Inference [23.65155934960922]
  Optimal control under uncertainty is a prevailing challenge in control, due to the difficulty in producing tractable solutions for the optimization problem. By framing the control problem as one of input estimation, advanced approximate inference techniques can be used to handle the statistical approximations in a principled and practical manner.
  arXiv  Detail & Related papers  (2021-05-17T09:27:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.