BayesFP: Posterior Estimation for Flow-Based Policies via Feynman-Kac Sampling

• URL: http://arxiv.org/abs/2606.21014v1
• Date: Fri, 19 Jun 2026 00:55:24 GMT
• Title: BayesFP: Posterior Estimation for Flow-Based Policies via Feynman-Kac Sampling
• Abstract summary: We develop a unified, inference-time, retraining-free sampler for flow-matching policies.<n>We show improvements on the base $_0.5 on zero-shot optimal tasks.
• Score: 7.25385467896207
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Robots must generate trajectories that remain faithful to learned expert behavior while satisfying safety constraints and task-specific objectives specified only at inference time. We formulate constrained trajectory generation for pretrained diffusion and flow-matching policies as Bayesian posterior sampling, with the learned demonstration distribution as a prior and an inference-time, cost-derived likelihood tilting it toward feasible, optimal trajectories. To sample from this posterior without any retraining of the base policy, we leverage the Feynman--Kac corrector framework, originally formulated for diffusion models, and extend it to deterministic flow-matching policies. The result is a unified, inference-time, retraining-free sampler for diffusion and flow policies. We validate the approach on pretrained Diffusion Policy, GR00T-N1.6, and $π_{0.5}$ checkpoints across simulated and real-world manipulation tasks, including planning around non-convex obstacles introduced at inference time, and show improvements over the base $π_{0.5}$ on zero-shot tasks.

Related papers

• Refinement-based Flow Policy Optimization [6.752424962104772]
  Refinement-Based Flow Policy Optimization (RFPO) is a novel framework for training a flow policy in online reinforcement learning.<n>RFPO incorporates Q-guidance into the policy without requiring direct samples from the target distribution.<n>RFPO captures complex multimodal structure without mode collapse.
  arXiv  Detail & Related papers  (2026-09-14T06:56:33Z)
• Post-Training at the Edge of Detectability: A Game-Theoretic Approach to Fine-Tuning [58.088509347353465]
  Reinforcement learning (RL) fine-tuning is widely used in language model training.<n>We propose a game-theoretic framework that gives this trade-off an explicit statistical interpretation.
  arXiv  Detail & Related papers  (2026-07-29T00:19:09Z)
• A Stabilized Path-Space Approach to Diffusion-Based Posterior Sampling [12.537644240443086]
  Diffusion models provide expressive data-driven priors for inverse problems.<n>Many diffusion posterior samplers rely on guidance approximations that can fail for nonlinear operators and multimodal posteriors.<n>We develop a stabilized path-space framework for diffusion-based posterior sampling.
  arXiv  Detail & Related papers  (2026-06-10T21:55:55Z)
• Stochastic MeanFlow Policies: One-Step Generative Control with Entropic Mirror Descent [53.828537014796574]
  Online off-policy reinforcement learning (RL) is shaped by two choices: the policy class and the update rule.<n>We propose MeanFlow Policies, a one-step generative policy class that maps noise to actions through a MeanFlow transformation.<n>Across seven MuJoCo benchmarks, S improves over Gaussian and generative baselines while retaining single-step inference efficiency.
  arXiv  Detail & Related papers  (2026-05-20T15:14:14Z)
• ProFlow: Zero-Shot Physics-Consistent Sampling via Proximal Flow Guidance [35.08166384258028]
  ProFlow is a framework for zero-shot physics-consistent sampling.<n>It reconciles strict physical consistency and observational fidelity with the statistical structure of the pre-trained prior.<n>It achieves superior physical and observational consistency, as well as more accurate distributional statistics.
  arXiv  Detail & Related papers  (2026-01-28T03:57:00Z)
• Reverse Flow Matching: A Unified Framework for Online Reinforcement Learning with Diffusion and Flow Policies [4.249024052507976]
  We propose a unified framework, reverse flow matching (RFM), which rigorously addresses the problem of training diffusion and flow models without direct target samples.<n>By adopting a reverse inferential perspective, we formulate the training target as a posterior mean estimation problem given an intermediate noisy sample.<n>We show that existing noise-expectation and gradient-expectation methods are two specific instances within this broader class.
  arXiv  Detail & Related papers  (2026-01-13T01:58:24Z)
• One-Step Flow Policy Mirror Descent [52.31612487608593]
  Flow Policy Mirror Descent (FPMD) is an online RL algorithm that enables 1-step sampling during flow policy inference.<n>Our approach exploits a theoretical connection between the distribution variance and the discretization error of single-step sampling in straight flow matching models.
  arXiv  Detail & Related papers  (2025-07-31T15:51:10Z)
• Inference-Time Scaling of Diffusion Language Models with Particle Gibbs Sampling [70.8832906871441]
  We study how to steer generation toward desired rewards without retraining the models.<n>Prior methods typically resample or filter within a single denoising trajectory, optimizing rewards step-by-step without trajectory-level refinement.<n>We introduce particle Gibbs sampling for diffusion language models (PG-DLM), a novel inference-time algorithm enabling trajectory-level refinement while preserving generation perplexity.
  arXiv  Detail & Related papers  (2025-07-11T08:00:47Z)
• Early Stopping in Contextual Bandits and Inferences [0.5439020425819]
  Excessive sampling can be costly, motivating the devolopment of early stopping methods and reliable post-experiment inferences.<n>This paper studies early stopping methods in linear contextual bandits, including both pre-determined and online stopping rules, to minimize in-experiment regrets while accounting for sampling costs.
  arXiv  Detail & Related papers  (2025-02-05T00:40:10Z)
• Rejection via Learning Density Ratios [50.91522897152437]
  Classification with rejection emerges as a learning paradigm which allows models to abstain from making predictions.<n>We propose a different distributional perspective, where we seek to find an idealized data distribution which maximizes a pretrained model's performance.<n>Our framework is tested empirically over clean and noisy datasets.
  arXiv  Detail & Related papers  (2024-05-29T01:32:17Z)
• Time-series Generation by Contrastive Imitation [87.51882102248395]
  We study a generative framework that seeks to combine the strengths of both: Motivated by a moment-matching objective to mitigate compounding error, we optimize a local (but forward-looking) transition policy. At inference, the learned policy serves as the generator for iterative sampling, and the learned energy serves as a trajectory-level measure for evaluating sample quality.
  arXiv  Detail & Related papers  (2023-11-02T16:45:25Z)
• Unsupervised Sampling Promoting for Stochastic Human Trajectory Prediction [10.717921532244613]
  We propose a novel method, called BOsampler, to adaptively mine potential paths with Bayesian optimization in an unsupervised manner. Specifically, we model the trajectory sampling as a Gaussian process and construct an acquisition function to measure the potential sampling value. This acquisition function applies the original distribution as prior and encourages exploring paths in the long-tail region.
  arXiv  Detail & Related papers  (2023-04-09T19:15:14Z)
• Distributional Reinforcement Learning via Moment Matching [54.16108052278444]
  We formulate a method that learns a finite set of statistics from each return distribution via neural networks. Our method can be interpreted as implicitly matching all orders of moments between a return distribution and its Bellman target. Experiments on the suite of Atari games show that our method outperforms the standard distributional RL baselines.
  arXiv  Detail & Related papers  (2020-07-24T05:18:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.