Language Generation as Optimal Control: Closed-Loop Diffusion in Latent Control Space

• URL: http://arxiv.org/abs/2605.14531v2
• Date: Fri, 15 May 2026 04:26:23 GMT
• Title: Language Generation as Optimal Control: Closed-Loop Diffusion in Latent Control Space
• Abstract summary: This work reformulates language generation as an optimal control problem.<n>It provides a unified theoretical perspective to analyze autoregressive and diffusion models.
• Score: 99.30250506439678
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This work reformulates language generation as a stochastic optimal control problem, providing a unified theoretical perspective to analyze autoregressive and diffusion models and explain their limitations (Efficiency-Fidelity Paradox, Irreversibility Error Propagation, Optimization Tractability and Fidelity) in terms of combination of trajectory singularity, adjoint state vanishing, and gradient absence. To address these issues, we approximate the solution to the Hamilton-Jacobi-Bellman (HJB) equation, yielding an optimal policy that acts as a closed-loop controller. To bypass the intractability of directly solving the HJB PDE, we employ Flow Matching as the optimal trajectory solver within the rectified latent control space. This allows our Manta-LM with Global Integral Operator to approximate the global vector field, effectively realizing a model that simultaneously achieves high-fidelity text generation and efficient, low-cost parallel sampling. Empirically, our method achieves strong performance on language modeling and conditional generation tasks, while exhibiting improved stability, efficiency, and controllability.

Related papers

• Grounding Generative Policies in Physics: Optimization-Guided Diffusion for Robot Control [41.782471938706344]
  For task-space robot policies, generated grasps, waypoints, or trajectories can be distributionally valid yet infeasible.<n>We propose an inference-time optimization framework that couples the behavior generation to physical feasibility.<n>We evaluate the method on dexterous grasp synthesis with reachability and collision-avoidance constraints, and dynamic manipulation with controller-level trackability constraints.
  arXiv  Detail & Related papers  (2026-06-23T06:49:30Z)
• ChainFlow-VLA: Causal Flow Planning with Vision-Language Models [7.814547378443577]
  Current end-to-end autonomous driving systems are fundamentally limited by a mismatch between temporal causal reasoning and global trajectory consistency.<n>We propose ChainFlow-VLA, which unifies causal generation and global refinement within a unified probabilistic framework.<n> Experiments demonstrate that ChainFlow-VLA achieves robust planning in ambiguous and long-tail scenarios, achieving a state-of-the-art score of 94.85 on the NAVSIM v1 leaderboard.
  arXiv  Detail & Related papers  (2026-05-22T06:17:35Z)
• Subspace Control: Turning Constrained Model Steering into Controllable Spectral Optimization [27.280175507074322]
  We propose a subspace control framework for constrained model training.<n>SIFT (spectral interference-free training) consistently achieves substantial and robust performance improvements across all tasks.
  arXiv  Detail & Related papers  (2026-04-05T19:27:24Z)
• Diffusion Controller: Framework, Algorithms and Parameterization [54.82539154511621]
  We present a unified control-theoretic view that casts reverse diffusion sampling as state-only control within (generalized) linearly-solvable Markov Decision Processes.<n>Under this framework, control acts by reweighting the pretrained reverse-time transition kernels, balancing terminal objectives against an $f$-divergence cost.<n>Experiments on Stable Diffusion v1.4 show consistent gains in preference-alignment win rates and improved quality-efficiency trade-offs.
  arXiv  Detail & Related papers  (2026-03-07T01:49:59Z)
• LFPO: Likelihood-Free Policy Optimization for Masked Diffusion Models [48.68246945083386]
  Likelihood-Free Policy Optimization (LFPO) is a native framework that maps the concept of vector field flow matching to the discrete token space.<n>LFPO formulates alignment as geometric velocity rectification, which directly optimize denoising logits via contrastive updates.<n>Experiments demonstrate that LFPO not only outperforms state-of-the-art baselines on code and reasoning benchmarks but also accelerates inference by approximately 20% through reduced diffusion steps.
  arXiv  Detail & Related papers  (2026-03-02T07:42:55Z)
• OmniVL-Guard: Towards Unified Vision-Language Forgery Detection and Grounding via Balanced RL [63.388513841293616]
  Existing forgery detection methods fail to handle the interleaved text, images, and videos prevalent in real-world misinformation.<n>To bridge this gap, this paper targets to develop a unified framework for omnibus vision-language forgery detection and grounding.<n>We propose textbf OmniVL-Guard, a balanced reinforcement learning framework for omnibus vision-language forgery detection and grounding.
  arXiv  Detail & Related papers  (2026-02-11T09:41:36Z)
• Variational Entropic Optimal Transport [67.76725267984578]
  We propose Variational Entropic Optimal Transport (VarEOT) for domain translation problems.<n>VarEOT is based on an exact variational reformulation of the log-partition $log mathbbE[exp(cdot)$ as a tractable generalization over an auxiliary positive normalizer.<n> Experiments on synthetic data and unpaired image-to-image translation demonstrate competitive or improved translation quality.
  arXiv  Detail & Related papers  (2026-02-02T15:48:44Z)
• Stochastic Control Methods for Optimization [0.0]
  In the Euclidean setting, we analyze the problem of regularized control problems.<n>For global measures, we formulate a regularized mean-field problem characterized by a master-field problem.
  arXiv  Detail & Related papers  (2026-01-03T17:55:26Z)
• Parallel Diffusion Solver via Residual Dirichlet Policy Optimization [88.7827307535107]
  Diffusion models (DMs) have achieved state-of-the-art generative performance but suffer from high sampling latency due to their sequential denoising nature.<n>Existing solver-based acceleration methods often face significant image quality degradation under a low-dimensional budget.<n>We propose the Ensemble Parallel Direction solver (dubbed as EPD-EPr), a novel ODE solver that mitigates these errors by incorporating multiple gradient parallel evaluations in each step.
  arXiv  Detail & Related papers  (2025-12-28T05:48:55Z)
• Learning based convex approximation for constrained parametric optimization [11.379408842026981]
  We propose an input neural network (ICNN)-based self-supervised learning framework to solve constrained optimization problems.<n>We provide rigorous convergence analysis, showing that the framework converges to a Karush-Kuhn-Tucker (KKT) approximation point of the original problem.<n>Our approach achieves a superior balance among accuracy, feasibility, and computational efficiency.
  arXiv  Detail & Related papers  (2025-05-07T00:33:14Z)
• Latent Bayesian Optimization via Autoregressive Normalizing Flows [17.063294409131238]
  We propose a Normalizing Flow-based Bayesian Optimization (NF-BO) to solve the value discrepancy problem.<n>Our method demonstrates superior performance in molecule generation tasks, significantly outperforming both traditional and recent LBO approaches.
  arXiv  Detail & Related papers  (2025-04-21T06:36:09Z)
• Stochastic Optimal Control Matching [53.156277491861985]
  Our work introduces Optimal Control Matching (SOCM), a novel Iterative Diffusion Optimization (IDO) technique for optimal control. The control is learned via a least squares problem by trying to fit a matching vector field. Experimentally, our algorithm achieves lower error than all the existing IDO techniques for optimal control.
  arXiv  Detail & Related papers  (2023-12-04T16:49:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.