VOiLA: Vectorized Online Planning with Learned Diffusion Model for POMDP Agents

• URL: http://arxiv.org/abs/2606.19729v1
• Date: Thu, 18 Jun 2026 02:51:54 GMT
• Title: VOiLA: Vectorized Online Planning with Learned Diffusion Model for POMDP Agents
• Abstract summary: Planning under uncertainty is an essential capability for autonomous robots.<n>We present a framework that learns task-agnostic POMDP models for online planning under uncertainty.
• Score: 9.270170611697141
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Planning under uncertainty is an essential capability for autonomous robots. The Partially Observable Markov Decision Process (POMDP) provides a powerful framework for such a capability. Although POMDP-based planning has advanced significantly, its application to real-world problems is often limited by the difficulty of obtaining faithful POMDP models. We present Vectorized Online planning wIth Learned diffusion model for POMDP Agents (VOiLA), a framework that learns task-agnostic POMDP models for online planning under uncertainty. VOiLA learns transition and observation samplers using conditional diffusion models and learns observation-likelihood models for particle-based belief updates. To enable efficient online planning, the diffusion samplers are distilled into compact feedforward generators and integrated with Vectorized Online POMDP Planner (VOPP), an online POMDP planner designed to leverage GPU parallelization. Experimental results indicate the distillation strategy reduces sampling cost by up to nearly three orders of magnitude, making learned generative POMDP models practical for online planning. Evaluation of VOiLA on three benchmark problems indicate that VOiLA achieves equal or better performance than Recurrent Soft Actor Critic while using less than 10% training data, and generalizes much better to unseen environment configurations. Physical robot evaluation indicates VOiLA uses the models learned using only simulated data and generates a policy that successfully accomplish the task in 10 of 10 runs.

Related papers

• Scalable Rao-Blackwellized Online Planning for High-Dimensional POMDPs [10.757434965980124]
  We extend the Rao-Blackwellized online POMDP (RB-POMDP) framework to improve its generalizability in high-dimensional settings through hybrid continuous-discrete belief representations.<n>We demonstrate the effectiveness of this framework in a robotic search-and-rescue task by integrating it with FastSLAM 2.0.
  arXiv  Detail & Related papers  (2026-09-01T14:57:10Z)
• Learning POMDP World Models from Observations with Language-Model Priors [52.11151769796862]
  Partially-observable Markov decision processes (POMDPs) provide a flexible modeling class for internal world models.<n>We ask whether language-model priors can reduce costly interaction by leveraging prior knowledge.<n>We show that emphPinductor matches the performance and sample efficiency of LLM-based POMDP learning methods.
  arXiv  Detail & Related papers  (2026-05-13T16:18:15Z)
• LLM-Guided Probabilistic Program Induction for POMDP Model Estimation [40.98644220584212]
  Partially Observable Markov Decision Processes (POMDPs) model decision making under uncertainty.<n>We are interested in a subclass of POMDPs wherein the components of the model, including the observation function, reward function, transition function, and initial state distribution function, can be modeled as low-complexity probabilistic graphical models.
  arXiv  Detail & Related papers  (2025-05-04T18:59:07Z)
• Latent Diffusion Planning for Imitation Learning [78.56207566743154]
  Latent Diffusion Planning (LDP) is a modular approach consisting of a planner and inverse dynamics model.<n>By separating planning from action prediction, LDP can benefit from the denser supervision signals of suboptimal and action-free data.<n>On simulated visual robotic manipulation tasks, LDP outperforms state-of-the-art imitation learning approaches.
  arXiv  Detail & Related papers  (2025-04-23T17:53:34Z)
• Learning Logic Specifications for Policy Guidance in POMDPs: an Inductive Logic Programming Approach [57.788675205519986]
  We learn high-quality traces from POMDP executions generated by any solver. We exploit data- and time-efficient Indu Logic Programming (ILP) to generate interpretable belief-based policy specifications. We show that learneds expressed in Answer Set Programming (ASP) yield performance superior to neural networks and similar to optimal handcrafted task-specifics within lower computational time.
  arXiv  Detail & Related papers  (2024-02-29T15:36:01Z)
• Simplifying Complex Observation Models in Continuous POMDP Planning with Probabilistic Guarantees and Practice [8.403582577557918]
  We deal with the question of what would be the implication of using simplified observation models for planning.<n>Our main contribution is a novel probabilistic bound based on a statistical total variation distance of the simplified model.<n>Our calculations can be separated into offline and online parts, and we arrive at formal guarantees without having to access the costly model at all during planning.
  arXiv  Detail & Related papers  (2023-11-13T20:55:02Z)
• Predictable MDP Abstraction for Unsupervised Model-Based RL [93.91375268580806]
  We propose predictable MDP abstraction (PMA) Instead of training a predictive model on the original MDP, we train a model on a transformed MDP with a learned action space. We theoretically analyze PMA and empirically demonstrate that PMA leads to significant improvements over prior unsupervised model-based RL approaches.
  arXiv  Detail & Related papers  (2023-02-08T07:37:51Z)
• Visual Learning-based Planning for Continuous High-Dimensional POMDPs [81.16442127503517]
  Visual Tree Search (VTS) is a learning and planning procedure that combines generative models learned offline with online model-based POMDP planning. VTS bridges offline model training and online planning by utilizing a set of deep generative observation models to predict and evaluate the likelihood of image observations in a Monte Carlo tree search planner. We show that VTS is robust to different observation noises and, since it utilizes online, model-based planning, can adapt to different reward structures without the need to re-train.
  arXiv  Detail & Related papers  (2021-12-17T11:53:31Z)
• Evaluating model-based planning and planner amortization for continuous control [79.49319308600228]
  We take a hybrid approach, combining model predictive control (MPC) with a learned model and model-free policy learning. We find that well-tuned model-free agents are strong baselines even for high DoF control problems. We show that it is possible to distil a model-based planner into a policy that amortizes the planning without any loss of performance.
  arXiv  Detail & Related papers  (2021-10-07T12:00:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.