Approximate Inference for Stochastic Planning in Factored Spaces

• URL: http://arxiv.org/abs/2203.12139v1
• Date: Wed, 23 Mar 2022 02:15:00 GMT
• Title: Approximate Inference for Stochastic Planning in Factored Spaces
• Authors: Zhennan Wu, Roni Khardon
• Abstract summary: The paper explores the use of approximate inference techniques as solution methods for planning problems with discrete factored spaces. We present a simple framework that captures and connects prior work along two dimensions, direction of information flow, and type of approximation used. We also propose a novel algorithm, CSVI, which provides a tighter variational approximation compared to prior work.
• Score: 6.467357887660512
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The paper explores the use of approximate inference techniques as solution methods for stochastic planning problems with discrete factored spaces. While much prior work exists on this topic, subtle variations hinder a global understanding of different approaches for their differences and potential advantages. Here we abstract a simple framework that captures and connects prior work along two dimensions, direction of information flow, i.e., forward vs backward inference, and the type of approximation used, e.g., Belief Propagation (BP) vs mean field variational inference (MFVI). Through this analysis we also propose a novel algorithm, CSVI, which provides a tighter variational approximation compared to prior work. An extensive experimental evaluation compares algorithms from different branches of the framework, showing that methods based on BP are generally better than methods based on MFVI, that CSVI is competitive with BP algorithms, and that while inference direction does not show a significant effect for VI methods, forward inference provides stronger performance with BP.

Related papers

• Stability-based Generalization Bounds for Variational Inference [3.146069168382982]
  Variational inference (VI) is widely used for approximate inference in Bayesian machine learning. This paper develops stability based generalization bounds for a class of approximate Bayesian algorithms. The new approach complements PAC-Bayes analysis and can provide tighter bounds in some cases.
  arXiv  Detail & Related papers  (2025-02-17T22:40:26Z)
• Optimistic Algorithms for Adaptive Estimation of the Average Treatment Effect [36.25361703897723]
  Recent advances in martingale theory have paved the way for adaptive methods that can enhance the power of downstream inference. We study adaptive sampling procedures that take advantage of optimalally optimal causal inference procedures. Our findings mark a step forward in advancing adaptive causal inference methods in theory and practice.
  arXiv  Detail & Related papers  (2025-02-07T05:39:32Z)
• A Trust-Region Method for Graphical Stein Variational Inference [3.5516599670943774]
  Stein variational (SVI) is a sample-based approximate inference technique that generates a sample set by jointly optimizing the samples locations to an information-theoretic measure. We propose a novel trust-conditioned approach for SVI that successfully addresses each these challenges.
  arXiv  Detail & Related papers  (2024-10-21T16:59:01Z)
• Embedding Trajectory for Out-of-Distribution Detection in Mathematical Reasoning [50.84938730450622]
  We propose a trajectory-based method TV score, which uses trajectory volatility for OOD detection in mathematical reasoning. Our method outperforms all traditional algorithms on GLMs under mathematical reasoning scenarios. Our method can be extended to more applications with high-density features in output spaces, such as multiple-choice questions.
  arXiv  Detail & Related papers  (2024-05-22T22:22:25Z)
• Provably Efficient UCB-type Algorithms For Learning Predictive State Representations [55.00359893021461]
  The sequential decision-making problem is statistically learnable if it admits a low-rank structure modeled by predictive state representations (PSRs) This paper proposes the first known UCB-type approach for PSRs, featuring a novel bonus term that upper bounds the total variation distance between the estimated and true models. In contrast to existing approaches for PSRs, our UCB-type algorithms enjoy computational tractability, last-iterate guaranteed near-optimal policy, and guaranteed model accuracy.
  arXiv  Detail & Related papers  (2023-07-01T18:35:21Z)
• Variance-Preserving-Based Interpolation Diffusion Models for Speech Enhancement [53.2171981279647]
  We present a framework that encapsulates both the VP- and variance-exploding (VE)-based diffusion methods. To improve performance and ease model training, we analyze the common difficulties encountered in diffusion models. We evaluate our model against several methods using a public benchmark to showcase the effectiveness of our approach.
  arXiv  Detail & Related papers  (2023-06-14T14:22:22Z)
• A Rigorous Link between Deep Ensembles and (Variational) Bayesian Methods [14.845158804951552]
  We establish the first mathematically rigorous link between Bayesian, variational Bayesian, and ensemble methods. On a technical level, our contribution amounts to a generalised variational inference through the lense of Wasserstein flows.
  arXiv  Detail & Related papers  (2023-05-24T11:13:59Z)
• A step towards the applicability of algorithms based on invariant causal learning on observational data [0.0]
  In this paper, we show how to apply Invariant Causal Prediction (ICP) efficiently integrated with causal discovery methods. We also show how to apply ICP efficiently integrated with our method for causal discovery.
  arXiv  Detail & Related papers  (2023-04-05T08:15:57Z)
• On Representations of Mean-Field Variational Inference [2.4316550366482357]
  We present a framework to analyze mean field variational inference (MFVI) algorithms. Our approach enables the MFVI problem to be represented in three different manners. Rigorous guarantees are established to show that a time-discretized implementation of the coordinate ascent variational inference algorithm yields a gradient flow in the limit.
  arXiv  Detail & Related papers  (2022-10-20T16:26:22Z)
• Error-Aware Spatial Ensembles for Video Frame Interpolation [50.63021118973639]
  Video frame(VFI) algorithms have improved considerably in recent years due to unprecedented progress in both data-driven algorithms and their implementations. Recent research has introduced advanced motion estimation or novel warping methods as the means to address challenging VFI scenarios. This work introduces such a solution. By closely examining the correlation between optical flow and IE, the paper proposes novel error prediction metrics that partition the middle frame into distinct regions corresponding to different IE levels.
  arXiv  Detail & Related papers  (2022-07-25T16:15:38Z)
• Inverse Reinforcement Learning with Explicit Policy Estimates [19.159290496678004]
  Various methods for solving the inverse reinforcement learning problem have been developed independently in machine learning and economics. We show that they all belong to a class of optimization problems, characterized by a common form of gradient, the associated policy and the objective. Using insights which emerge from our study of this class of optimization problems, we identify various problem scenarios and investigate each method's suitability for these problems.
  arXiv  Detail & Related papers  (2021-03-04T07:00:58Z)
• Semi-Supervised Learning with Variational Bayesian Inference and Maximum Uncertainty Regularization [62.21716612888669]
  We propose two generic methods for improving semi-supervised learning (SSL) The first integrates weight perturbation (WP) into existing "consistency regularization" (CR) based methods. The second method proposes a novel consistency loss called "maximum uncertainty regularization" (MUR)
  arXiv  Detail & Related papers  (2020-12-03T09:49:35Z)
• Learning explanations that are hard to vary [75.30552491694066]
  We show that averaging across examples can favor memorization and patchwork' solutions that sew together different strategies. We then propose and experimentally validate a simple alternative algorithm based on a logical AND.
  arXiv  Detail & Related papers  (2020-09-01T10:17:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.