Guided Streaming Stochastic Interpolant Policy

• URL: http://arxiv.org/abs/2605.10051v1
• Date: Mon, 11 May 2026 06:24:21 GMT
• Title: Guided Streaming Stochastic Interpolant Policy
• Abstract summary: Inference-time guidance is essential for steering generative robot policies toward dynamic objectives without retraining.<n>In this work, we derive the optimal guidance term for dynamic, unstructured Interpolants (SI)<n>We apply this framework to real-time control through the Streaming Interpolant Policy (SSIP)
• Score: 8.673777984856075
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Inference-time guidance is essential for steering generative robot policies toward dynamic objectives without retraining, yet existing methods are largely confined to chunk-based architectures that exhibit high latency and lack the reactivity needed for test-time preference alignment or obstacle avoidance. In this work, we formally derive the optimal guidance term for Stochastic Interpolants (SI) by analyzing the value function's time evolution via the Backward Kolmogorov Equation, establishing a modified drift that theoretically guarantees sampling from a target distribution. We apply this framework to real-time control through the Streaming Stochastic Interpolant Policy (SSIP), which generalizes the deterministic Streaming Flow Policy (SFP). Unifying this guidance law with the streaming architecture enables fast and reactive control. To support diverse deployment needs, we propose two complementary mechanisms: training-free Stochastic Trajectory Ensemble Guidance (STEG) that computes gradients on-the-fly for zero-shot adaptation, and training-based Conditional Critic Guidance (CCG) for amortized inference. Empirical evaluations demonstrate that our guided streaming approach significantly outperforms conventional chunk-based policies in reactivity and provides superior, physically valid guidance for dynamic, unstructured environments.

Related papers

• Path Integral Value Matching for Linear Quadratic Stochastic Optimal Control [26.034302137738127]
  Current state-of-the-art policy-based methods suffer from prohibitive computational costs and instability.<n>We propose a paradigm shift toward a value-based approach by revisiting Path Integral Control (PIC)<n>Building upon this theoretical foundation, we propose the Path Integral Value Matching (PI-VM) algorithm.
  arXiv  Detail & Related papers  (2026-08-11T10:29:49Z)
• IADD-TR: Intervention-Aware Dynamics Decoupling with Targeted Regularization for Model-Based Reinforcement Learning [57.16513298507272]
  We propose IADD-TR, a unified framework combining Intervention-Aware Dynamics Decoupling (IADD) and Targeted Regularization (TR)<n>IADD factorizes transitions into an action-intervention stage and an action-free natural evolution stage, using a zero-action anchor to resolve the non-uniqueness of this two-stage factorization for robust generalization.<n>For policy learning, we derive TR from the efficient influence function of a replay-state policy-gradient functional.
  arXiv  Detail & Related papers  (2026-08-11T08:20:02Z)
• Test-Time Gradient Guidance of Flow Policies in Reinforcement Learning [50.738952715864116]
  Expressive continuous control policies form the backbone of advances in scaling imitation learning for simulated and real robot control.<n>We propose QGF (Q-Guided Flow), an RL algorithm that performs policy optimization entirely at test time.<n> Empirically, QGF outperforms prior test-time RL methods on single-task and goal-conditioned offline RL benchmarks.
  arXiv  Detail & Related papers  (2026-06-09T16:45:57Z)
• Iterative Refinement of Flow Policies in Probability Space for Online Reinforcement Learning [56.47948583452555]
  We introduce the Stepwise Flow Policy (SWFP) framework, founded on the key insight that discretizing the flow matching inference process via a fixed-step Euler scheme aligns it with the variational Jordan-Kinderlehrer-Otto principle from optimal transport.<n>SWFP decomposes the global flow into a sequence of small, incremental transformations between proximate distributions.<n>This decomposition yields an efficient algorithm that fine-tunes pre-trained flows via a cascade of small flow blocks, offering significant advantages.
  arXiv  Detail & Related papers  (2025-10-17T07:43:51Z)
• Distributionally Robust Control with End-to-End Statistically Guaranteed Metric Learning [5.309590159815129]
  We propose a novel end-to-end finite-horizon Wasserstein DRC framework.<n>It integrates the learning of anisotropic Wasserstein metrics with downstream control tasks in a closed-loop manner.<n>We show that the proposed framework achieves superior closed-loop performance and robustness compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-10-11T13:40:49Z)
• Stabilizing Policy Gradients for Sample-Efficient Reinforcement Learning in LLM Reasoning [77.92320830700797]
  Reinforcement Learning has played a central role in enabling reasoning capabilities of Large Language Models.<n>We propose a tractable computational framework that tracks and leverages curvature information during policy updates.<n>The algorithm, Curvature-Aware Policy Optimization (CAPO), identifies samples that contribute to unstable updates and masks them out.
  arXiv  Detail & Related papers  (2025-10-01T12:29:32Z)
• SAC Flow: Sample-Efficient Reinforcement Learning of Flow-Based Policies via Velocity-Reparameterized Sequential Modeling [9.936731043466699]
  Training expressive flow-based policies with off-policy reinforcement learning is notoriously unstable due to gradient pathologies in the multi-step action sampling process.<n>We trace this instability to a fundamental connection: the flow rollout is algebraically equivalent to a residual recurrent computation, making it susceptible to the same vanishing and exploding gradients as RNNs.<n>We develop a practical SAC-based algorithm, enabled by a noise-augmented rollout, that facilitates direct end-to-end training of these policies.
  arXiv  Detail & Related papers  (2025-09-30T04:21:20Z)
• Extending Group Relative Policy Optimization to Continuous Control: A Theoretical Framework for Robotic Reinforcement Learning [0.0]
  Group Relative Policy Optimization (GRPO) has shown promise in discrete action spaces by eliminating value function dependencies through group-based advantage estimation.<n>This paper presents a theoretical framework extending GRPO to continuous control environments, addressing challenges in high-dimensional action spaces, sparse rewards, and temporal dynamics.
  arXiv  Detail & Related papers  (2025-07-25T05:25:40Z)
• Relative Entropy Pathwise Policy Optimization [66.03329137921949]
  We present an on-policy algorithm that trains Q-value models purely from on-policy trajectories.<n>We show how to combine policies for exploration with constrained updates for stable training, and evaluate important architectural components that stabilize value function learning.
  arXiv  Detail & Related papers  (2025-07-15T06:24:07Z)
• Logarithmic Smoothing for Adaptive PAC-Bayesian Off-Policy Learning [4.48890356952206]
  Off-policy learning serves as the primary framework for learning optimal policies from logged interactions.<n>We extend this framework to the adaptive scenario using tools from online PAC-Bayesian theory.
  arXiv  Detail & Related papers  (2025-06-12T12:54:09Z)
• A Regularized Implicit Policy for Offline Reinforcement Learning [54.7427227775581]
  offline reinforcement learning enables learning from a fixed dataset, without further interactions with the environment. We propose a framework that supports learning a flexible yet well-regularized fully-implicit policy. Experiments and ablation study on the D4RL dataset validate our framework and the effectiveness of our algorithmic designs.
  arXiv  Detail & Related papers  (2022-02-19T20:22:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.