Scalable Rao-Blackwellized Online Planning for High-Dimensional POMDPs

• URL: http://arxiv.org/abs/2609.01351v1
• Date: Tue, 01 Sep 2026 14:57:10 GMT
• Title: Scalable Rao-Blackwellized Online Planning for High-Dimensional POMDPs
• Abstract summary: We extend the Rao-Blackwellized online POMDP (RB-POMDP) framework to improve its generalizability in high-dimensional settings through hybrid continuous-discrete belief representations.<n>We demonstrate the effectiveness of this framework in a robotic search-and-rescue task by integrating it with FastSLAM 2.0.
• Score: 10.757434965980124
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Online planning under uncertainty remains a fundamental challenge for robotic systems operating in partially observable environments with high-dimensional state spaces. While sampling-based POMDP solvers enable approximate decision-making in large or continuous domains, their performance degrades as belief dimensionality increases due to the high variance inherent in Monte Carlo-based estimation. In this work, we extend the Rao-Blackwellized online POMDP (RB-POMDP) framework to improve its generalizability in high-dimensional settings through hybrid continuous-discrete belief representations. By analytically propagating uncertainty associated with marginalized state components during tree-based planning, the proposed approach reduces sampling-induced variance in value estimation. We demonstrate the effectiveness of this framework in a robotic search-and-rescue task by integrating it with FastSLAM 2.0. Experimental results show that the proposed planner achieves higher cumulative rewards using significantly fewer particles and planning simulations than purely sampling-based methods under equivalent computational budgets. These results suggest that structured high-dimensional robotic problems admitting tractable sufficient statistics can be effectively leveraged within the RB-POMDP framework for computationally feasible online decision-making.

Related papers

• Case-Guided Sequential Assay Planning in Drug Discovery [2.8529443025686487]
  Implicit Bayesian Markov Decision Process (IBMDP) is a model-based RL framework designed for simulator-free settings.<n>IBMDP generates stable policies that balance information gain toward desired outcomes with resource efficiency.<n>On a real-world central nervous system (CNS) drug discovery task, IBMDP reduced resource consumption by up to 92% compared to establisheds.
  arXiv  Detail & Related papers  (2026-01-21T06:58:01Z)
• Online Robust Planning under Model Uncertainty: A Sample-Based Approach [8.599681538174888]
  We introduce Robust Sparse Sampling ( RSS), the first online planning algorithm for Markov Decision Processes (MDPs) with finite-sample theoretical performance guarantees.<n> RSS computes a robust value function by leveraging the efficiency and theoretical properties of Sample Average Approximation (SAA)<n> RSS is applicable to infinite or continuous state spaces, and its sample and computational complexities are independent of the state space size.
  arXiv  Detail & Related papers  (2025-09-12T11:41:23Z)
• Solving Finite-Horizon MDPs via Low-Rank Tensors [9.072279909866845]
  We study the problem of learning optimal policies in finite-horizon Markov Decision Processes (MDPs)<n>In finite-horizon MDPs, the policies, and therefore the value functions (VFs) are not stationary.<n>We propose modeling the VFs of finite-horizon MDPs as low-rank tensors, enabling a scalable representation that renders the problem of learning optimal policies tractable.
  arXiv  Detail & Related papers  (2025-01-17T23:10:50Z)
• Rao-Blackwellized POMDP Planning [10.762101459838048]
  This study introduces Rao-Blackwellized POMDP approximate solvers and outlines generic methods to apply Rao-Blackwellization in both belief updates and online planning.<n>We compare the performance of SIRPF and RBPF in a simulated localization problem where an agent navigates toward a target in a GPS-denied environment using POMCPOW and RB-POMCPOW planners.<n>Our results not only confirm that RBPFs maintain accurate belief approximations over time with fewer particles, but, more surprisingly, RBPFs combined with quadrature-based integration improve planning quality significantly compared to SIRPF-based planning under
  arXiv  Detail & Related papers  (2024-09-24T18:46:50Z)
• Improved Monte Carlo Planning via Causal Disentanglement for Structurally-Decomposed Markov Decision Processes [0.9768138268100163]
  We introduce the Structurally Decomposed MDP (SD-MDP), which leverages causal disentanglement to partition an MDP's temporal causal graph into independent components.<n>We demonstrate superior policy performance over benchmarks across various logistics and finance domains.
  arXiv  Detail & Related papers  (2024-06-23T16:22:40Z)
• Sample Complexity of Offline Distributionally Robust Linear Markov Decision Processes [37.15580574143281]
  offline reinforcement learning (RL) This paper considers the sample complexity of distributionally robust linear Markov decision processes (MDPs) with an uncertainty set characterized by the total variation distance using offline data. We develop a pessimistic model-based algorithm and establish its sample complexity bound under minimal data coverage assumptions.
  arXiv  Detail & Related papers  (2024-03-19T17:48:42Z)
• Learning Logic Specifications for Policy Guidance in POMDPs: an Inductive Logic Programming Approach [57.788675205519986]
  We learn high-quality traces from POMDP executions generated by any solver. We exploit data- and time-efficient Indu Logic Programming (ILP) to generate interpretable belief-based policy specifications. We show that learneds expressed in Answer Set Programming (ASP) yield performance superior to neural networks and similar to optimal handcrafted task-specifics within lower computational time.
  arXiv  Detail & Related papers  (2024-02-29T15:36:01Z)
• Provably Efficient UCB-type Algorithms For Learning Predictive State Representations [55.00359893021461]
  The sequential decision-making problem is statistically learnable if it admits a low-rank structure modeled by predictive state representations (PSRs) This paper proposes the first known UCB-type approach for PSRs, featuring a novel bonus term that upper bounds the total variation distance between the estimated and true models. In contrast to existing approaches for PSRs, our UCB-type algorithms enjoy computational tractability, last-iterate guaranteed near-optimal policy, and guaranteed model accuracy.
  arXiv  Detail & Related papers  (2023-07-01T18:35:21Z)
• The Curious Price of Distributional Robustness in Reinforcement Learning with a Generative Model [71.59406356321101]
  This paper investigates model robustness in reinforcement learning (RL) to reduce the sim-to-real gap in practice.<n>We adopt the framework of distributionally robust Markov decision processes (RMDPs), aimed at learning a policy that optimize the worst-case performance when the deployed environment falls within a prescribed uncertainty set around the nominal MDP.
  arXiv  Detail & Related papers  (2023-05-26T02:32:03Z)
• When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework for Offline Inverse Reinforcement Learning [62.00672284480755]
  This paper aims to recover the structure of rewards and environment dynamics that underlie observed actions in a fixed, finite set of demonstrations from an expert agent. Accurate models of expertise in executing a task has applications in safety-sensitive applications such as clinical decision making and autonomous driving.
  arXiv  Detail & Related papers  (2023-02-15T04:14:20Z)
• Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logic [59.94347858883343]
  This paper investigates the motion planning of autonomous dynamical systems modeled by Markov decision processes (MDP) The novelty is to design an embedded product MDP (EP-MDP) between the LDGBA and the MDP. The proposed LDGBA-based reward shaping and discounting schemes for the model-free reinforcement learning (RL) only depend on the EP-MDP states.
  arXiv  Detail & Related papers  (2021-02-24T01:11:25Z)
• Decentralized MCTS via Learned Teammate Models [89.24858306636816]
  We present a trainable online decentralized planning algorithm based on decentralized Monte Carlo Tree Search. We show that deep learning and convolutional neural networks can be employed to produce accurate policy approximators.
  arXiv  Detail & Related papers  (2020-03-19T13:10:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.