SNAPO: Smooth Neural Adjoint Policy Optimization for Optimal Control via Differentiable Simulation

• URL: http://arxiv.org/abs/2605.06570v1
• Date: Thu, 07 May 2026 17:01:13 GMT
• Title: SNAPO: Smooth Neural Adjoint Policy Optimization for Optimal Control via Differentiable Simulation
• Abstract summary: SNAPO is a framework that embeds a neural policy inside a known, differentiable simulator.<n>It computes exact gradients of the objective with respect to all policy parameters and all inputs in a single adjoint pass.<n>We demonstrate SNAPO on three domains: natural gas storage (training in under a minute, 365 forward curve sensitivities at no additional cost per sensitivity), pension fund asset-liability management, and pharmaceutical manufacturing.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Many real-world problems require sequential decisions under uncertainty: when to inject or withdraw gas from storage, how to rebalance a pension portfolio each month, what temperature profile to run through a pharmaceutical reactor chain. Dynamic programming solves small instances exactly but scales exponentially in state dimensions. Black-box reinforcement learning handles high-dimensional states but trains slowly and produces no sensitivities. We introduce SNAPO (Smooth Neural Adjoint Policy Optimization), a framework that embeds a neural policy inside a known, differentiable simulator, replaces hard constraints with smooth approximations, and computes exact gradients of the objective with respect to all policy parameters and all inputs in a single adjoint pass. We demonstrate SNAPO on three domains: natural gas storage (training in under a minute, 365 forward curve sensitivities at no additional cost per sensitivity), pension fund asset-liability management (6.5x-200x sensitivity speedup over bump-and-revalue, scaling with the number of risk factors), and pharmaceutical manufacturing (cross-unit sensitivities through a 4-unit process chain, with 20 ICH Q8 regulatory sensitivities from 5 adjoint passes in 74.5 milliseconds). All sensitivities are produced by the same backward pass that trains the policy, at a cost proportional to one reverse pass regardless of how many sensitivities are computed.

Related papers

• SS-ESOAP: Self-Scaled Adaptive Preconditioning for Physics-Informed Learning [70.71795707610961]
  method augments SOAP-style preconditioning with a scalar secant-energy correction adapted to Kronecker geometry.<n>Method attains the lowest final residual on six of eight PDE benchmarks.<n>Three-seed $L2$ and $H1$ errors on four representative PDEs support the link between lower residuals and improved solution accuracy.
  arXiv  Detail & Related papers  (2026-08-29T21:34:47Z)
• Spectral Pre-Filtering for Context-Adaptive Sensor Fusion: A Four-Role FFT-GDCB Integration for High-Stakes Decision Systems [0.0]
  Context-adaptive Kalman filters calibrate their noise covariance matrices Q and R from innovation residuals via online regression.<n>We introduce a four-role FFT pre-filter that solves this problem at $O(Nlog N)$ cost and serves three additional roles "for free"<n>The single $O(Nlog N)$ FFT call thereby serves four downstream consumers, fits in 0.1% of the sensor-fusion or pricing-pipeline compute budget, and is a drop-in addition with no changes to the Kalman filter, bandit, or runtime composition operator.
  arXiv  Detail & Related papers  (2026-08-22T15:57:25Z)
• An Embedded RISC-V Evaluation of Kolmogorov--Arnold Networks in Hard-Constrained Recurrent Physics-Informed Models [0.0]
  Kolmogorov--Arnold Networks (KANs) have been proposed as efficient replacements for multilayer perceptrons (MLPs)<n>This paper asks whether that parameter efficiency survives deployment.
  arXiv  Detail & Related papers  (2026-08-01T16:20:24Z)
• Trust the Batch, On- or Off-Policy: Adaptive Policy Optimization for RL Post-Training [50.86545293331458]
  Reinforcement learning is structurally harder than supervised learning because the policy changes the data distribution it learns from.<n>We present a simple yet effective batch-adaptive objective that replaces fixed clipping with the normalized effective sample size of the policy ratios.
  arXiv  Detail & Related papers  (2026-05-12T16:44:47Z)
• Training Non-Differentiable Networks via Optimal Transport [1.5229257192293202]
  PolyStep is a gradient-free method that updates parameters using only forward passes.<n>We train genuinely non-differentiable models where existing gradient-free methods collapse to near-random accuracy.<n>On hard-LIF spiking networks we reach 93.4% test accuracy, outperforming all gradient-free baselines by over 60pp and closing to within 4.4pp of a surrogate-gradient Adam ceiling.
  arXiv  Detail & Related papers  (2026-05-03T15:20:56Z)
• Anatomical Token Uncertainty for Transformer-Guided Active MRI Acquisition [27.825869252861335]
  Full data acquisition in MRI is inherently slow, which limits clinical throughput and increases patient discomfort.<n>CSMRI seeks to accelerate acquisition by reconstructing images from under-sampled kspace data.
  arXiv  Detail & Related papers  (2026-03-23T10:49:23Z)
• PolyGLU: State-Conditional Activation Routing in Transformer Feed-Forward Networks [0.0]
  We introduce PolyGLU, a drop-in replacement for SwiGLU that enables each FFN neuron to dynamically route among K=4 activation functions.<n>On standard benchmarks, PolychromaticLM achieves 62-89% of Qwen3-0.6B-Base performance despite training on 3,600x fewer tokens.
  arXiv  Detail & Related papers  (2026-03-07T10:39:56Z)
• Operator Learning Using Weak Supervision from Walk-on-Spheres [81.26322147849918]
  Training neural PDE solvers is often bottlenecked by expensive data generation or unstable physics-informed neural network (PINN)<n>We propose an alternative approach using Monte Carlo approaches to estimate the solution to the PDE as a process for weak supervision during training.
  arXiv  Detail & Related papers  (2026-03-01T17:23:39Z)
• ODAR: Principled Adaptive Routing for LLM Reasoning via Active Inference [60.958331943869126]
  ODAR-Expert is an adaptive routing framework that optimize the accuracy-efficiency trade-off via principled resource allocation.<n>We show strong and consistent gains, including 98.2% accuracy on MATH and 54.8% on Humanity's Last Exam.
  arXiv  Detail & Related papers  (2026-02-27T05:22:01Z)
• Neural Policy Iteration for Stochastic Optimal Control: A Physics-Informed Approach [2.8988658640181826]
  We propose a physics-informed neural network policy iteration framework (PINN-PI)<n>At each iteration, a neural network is trained to approximate the value function by minimizing the residual of a linear PDE induced by a fixed policy.<n>We demonstrate the effectiveness of our method on several benchmark problems, including gradient cartpole, pendulum high-dimensional linear quadratic regulation (LQR) problems in up to 10D.
  arXiv  Detail & Related papers  (2025-08-03T11:02:25Z)
• DeepRV: Accelerating spatiotemporal inference with pre-trained neural priors [0.0]
  We introduce DeepRV, a neural-network surrogate that closely matches full GP accuracy including hyperRV estimates.<n>Across simulated benchmarks, DeepRV achieves non-separable programming, and a real-world application to education deprivation in London.<n>All experiments run on a single consumer-grade GPU to ensure accessibility for practitioners.
  arXiv  Detail & Related papers  (2025-03-27T13:04:41Z)
• Message Passing Neural PDE Solvers [60.77761603258397]
  We build a neural message passing solver, replacing allally designed components in the graph with backprop-optimized neural function approximators. We show that neural message passing solvers representationally contain some classical methods, such as finite differences, finite volumes, and WENO schemes. We validate our method on various fluid-like flow problems, demonstrating fast, stable, and accurate performance across different domain topologies, equation parameters, discretizations, etc., in 1D and 2D.
  arXiv  Detail & Related papers  (2022-02-07T17:47:46Z)
• Softmax Policy Gradient Methods Can Take Exponential Time to Converge [60.98700344526674]
  The softmax policy gradient (PG) method is arguably one of the de facto implementations of policy optimization in modern reinforcement learning. We demonstrate that softmax PG methods can take exponential time -- in terms of $mathcalS|$ and $frac11-gamma$ -- to converge.
  arXiv  Detail & Related papers  (2021-02-22T18:56:26Z)
• GradInit: Learning to Initialize Neural Networks for Stable and Efficient Training [59.160154997555956]
  We present GradInit, an automated and architecture method for initializing neural networks. It is based on a simple agnostic; the variance of each network layer is adjusted so that a single step of SGD or Adam results in the smallest possible loss value. It also enables training the original Post-LN Transformer for machine translation without learning rate warmup.
  arXiv  Detail & Related papers  (2021-02-16T11:45:35Z)
• Kalman meets Bellman: Improving Policy Evaluation through Value Tracking [59.691919635037216]
  Policy evaluation is a key process in Reinforcement Learning (RL) We devise an optimization method, called Kalman Optimization for Value Approximation (KOVA) KOVA minimizes a regularized objective function that concerns both parameter and noisy return uncertainties.
  arXiv  Detail & Related papers  (2020-02-17T13:30:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.