A Bayesian Approach to Online Planning

• URL: http://arxiv.org/abs/2406.02103v1
• Date: Tue, 4 Jun 2024 08:33:17 GMT
• Title: A Bayesian Approach to Online Planning
• Authors: Nir Greshler, David Ben Eli, Carmel Rabinovitz, Gabi Guetta, Liran Gispan, Guy Zohar, Aviv Tamar,
• Abstract summary: Combination of Monte Carlo tree search and neural networks has revolutionized online planning. We ask whether uncertainty estimates about the network outputs could be used to improve planning. We develop a Bayesian planning approach that facilitates such uncertainty quantification, inspired by classical ideas from the meta-reasoning literature.
• Score: 14.847090489758992
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The combination of Monte Carlo tree search and neural networks has revolutionized online planning. As neural network approximations are often imperfect, we ask whether uncertainty estimates about the network outputs could be used to improve planning. We develop a Bayesian planning approach that facilitates such uncertainty quantification, inspired by classical ideas from the meta-reasoning literature. We propose a Thompson sampling based algorithm for searching the tree of possible actions, for which we prove the first (to our knowledge) finite time Bayesian regret bound, and propose an efficient implementation for a restricted family of posterior distributions. In addition we propose a variant of the Bayes-UCB method applied to trees. Empirically, we demonstrate that on the ProcGen Maze and Leaper environments, when the uncertainty estimates are accurate but the neural network output is inaccurate, our Bayesian approach searches the tree much more effectively. In addition, we investigate whether popular uncertainty estimation methods are accurate enough to yield significant gains in planning. Our code is available at: https://github.com/nirgreshler/bayesian-online-planning.

Related papers

• Depth-Bounded Epistemic Planning [50.42592219248395]
  We present a novel algorithm for planning based on dynamic epistemic logic. The novelty is that we limit the depth of reasoning of the planning agent to an upper bound b. We show it to be complete with respect to planning tasks having a solution within bound b of reasoning depth.
  arXiv  Detail & Related papers  (2024-06-03T09:30:28Z)
• Bayesian Online Natural Gradient (BONG) [9.800443064368467]
  We propose a novel approach to sequential Bayesian inference based on variational Bayes (VB) The key insight is that, in the online setting, we do not need to add the KL term to regularize to the prior. We show empirically that our method outperforms other online VB methods in the non-conjugate setting.
  arXiv  Detail & Related papers  (2024-05-30T04:27:36Z)
• Model-Free Local Recalibration of Neural Networks [0.0]
  Uncalibrated probabilistic forecasts are of limited use for many important decision-making tasks. We propose a localized recalibration of ANN predictive distributions using the dimension-reduced representation of the input. We show that our method has good performance compared to alternative approaches.
  arXiv  Detail & Related papers  (2024-03-09T01:58:45Z)
• Learning Active Subspaces for Effective and Scalable Uncertainty Quantification in Deep Neural Networks [13.388835540131508]
  We propose a novel scheme for constructing a low-dimensional subspace of the neural network parameters. We demonstrate that the significantly reduced active subspace enables effective and scalable Bayesian inference. Our approach provides reliable predictions with robust uncertainty estimates for various regression tasks.
  arXiv  Detail & Related papers  (2023-09-06T15:00:36Z)
• Semantic Strengthening of Neuro-Symbolic Learning [85.6195120593625]
  Neuro-symbolic approaches typically resort to fuzzy approximations of a probabilistic objective. We show how to compute this efficiently for tractable circuits. We test our approach on three tasks: predicting a minimum-cost path in Warcraft, predicting a minimum-cost perfect matching, and solving Sudoku puzzles.
  arXiv  Detail & Related papers  (2023-02-28T00:04:22Z)
• Improved uncertainty quantification for neural networks with Bayesian last layer [0.0]
  Uncertainty quantification is an important task in machine learning. We present a reformulation of the log-marginal likelihood of a NN with BLL which allows for efficient training using backpropagation.
  arXiv  Detail & Related papers  (2023-02-21T20:23:56Z)
• Layer Ensembles [95.42181254494287]
  We introduce a method for uncertainty estimation that considers a set of independent categorical distributions for each layer of the network. We show that the method can be further improved by ranking samples, resulting in models that require less memory and time to run.
  arXiv  Detail & Related papers  (2022-10-10T17:52:47Z)
• BayesCap: Bayesian Identity Cap for Calibrated Uncertainty in Frozen Neural Networks [50.15201777970128]
  We propose BayesCap that learns a Bayesian identity mapping for the frozen model, allowing uncertainty estimation. BayesCap is a memory-efficient method that can be trained on a small fraction of the original dataset. We show the efficacy of our method on a wide variety of tasks with a diverse set of architectures.
  arXiv  Detail & Related papers  (2022-07-14T12:50:09Z)
• Robustification of Online Graph Exploration Methods [59.50307752165016]
  We study a learning-augmented variant of the classical, notoriously hard online graph exploration problem. We propose an algorithm that naturally integrates predictions into the well-known Nearest Neighbor (NN) algorithm.
  arXiv  Detail & Related papers  (2021-12-10T10:02:31Z)
• Bayesian Optimization Meets Laplace Approximation for Robotic Introspection [41.117361086267806]
  We introduce a scalable Laplace Approximation (LA) technique to make Deep Neural Networks (DNNs) more introspective. In particular, we propose a novel Bayesian Optimization (BO) algorithm to mitigate their tendency of under-fitting the true weight posterior. We show that the proposed framework can be scaled up to large datasets and architectures.
  arXiv  Detail & Related papers  (2020-10-30T09:28:10Z)
• ESPN: Extremely Sparse Pruned Networks [50.436905934791035]
  We show that a simple iterative mask discovery method can achieve state-of-the-art compression of very deep networks. Our algorithm represents a hybrid approach between single shot network pruning methods and Lottery-Ticket type approaches.
  arXiv  Detail & Related papers  (2020-06-28T23:09:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.