Approximate Inference for Stochastic Planning in Factored Spaces
- URL: http://arxiv.org/abs/2203.12139v1
- Date: Wed, 23 Mar 2022 02:15:00 GMT
- Title: Approximate Inference for Stochastic Planning in Factored Spaces
- Authors: Zhennan Wu, Roni Khardon
- Abstract summary: The paper explores the use of approximate inference techniques as solution methods for planning problems with discrete factored spaces.
We present a simple framework that captures and connects prior work along two dimensions, direction of information flow, and type of approximation used.
We also propose a novel algorithm, CSVI, which provides a tighter variational approximation compared to prior work.
- Score: 6.467357887660512
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The paper explores the use of approximate inference techniques as solution
methods for stochastic planning problems with discrete factored spaces. While
much prior work exists on this topic, subtle variations hinder a global
understanding of different approaches for their differences and potential
advantages. Here we abstract a simple framework that captures and connects
prior work along two dimensions, direction of information flow, i.e., forward
vs backward inference, and the type of approximation used, e.g., Belief
Propagation (BP) vs mean field variational inference (MFVI). Through this
analysis we also propose a novel algorithm, CSVI, which provides a tighter
variational approximation compared to prior work. An extensive experimental
evaluation compares algorithms from different branches of the framework,
showing that methods based on BP are generally better than methods based on
MFVI, that CSVI is competitive with BP algorithms, and that while inference
direction does not show a significant effect for VI methods, forward inference
provides stronger performance with BP.
Related papers
- Trajectory Volatility for Out-of-Distribution Detection in Mathematical Reasoning [50.84938730450622]
We propose a trajectory-based method TV score, which uses trajectory volatility for OOD detection in mathematical reasoning.
Our method outperforms all traditional algorithms on GLMs under mathematical reasoning scenarios.
Our method can be extended to more applications with high-density features in output spaces, such as multiple-choice questions.
arXiv Detail & Related papers (2024-05-22T22:22:25Z) - Provably Efficient UCB-type Algorithms For Learning Predictive State
Representations [55.00359893021461]
The sequential decision-making problem is statistically learnable if it admits a low-rank structure modeled by predictive state representations (PSRs)
This paper proposes the first known UCB-type approach for PSRs, featuring a novel bonus term that upper bounds the total variation distance between the estimated and true models.
In contrast to existing approaches for PSRs, our UCB-type algorithms enjoy computational tractability, last-iterate guaranteed near-optimal policy, and guaranteed model accuracy.
arXiv Detail & Related papers (2023-07-01T18:35:21Z) - Variance-Preserving-Based Interpolation Diffusion Models for Speech
Enhancement [53.2171981279647]
We present a framework that encapsulates both the VP- and variance-exploding (VE)-based diffusion methods.
To improve performance and ease model training, we analyze the common difficulties encountered in diffusion models.
We evaluate our model against several methods using a public benchmark to showcase the effectiveness of our approach.
arXiv Detail & Related papers (2023-06-14T14:22:22Z) - A Rigorous Link between Deep Ensembles and (Variational) Bayesian
Methods [14.845158804951552]
We establish the first mathematically rigorous link between Bayesian, variational Bayesian, and ensemble methods.
On a technical level, our contribution amounts to a generalised variational inference through the lense of Wasserstein flows.
arXiv Detail & Related papers (2023-05-24T11:13:59Z) - A step towards the applicability of algorithms based on invariant causal
learning on observational data [0.0]
In this paper, we show how to apply Invariant Causal Prediction (ICP) efficiently integrated with causal discovery methods.
We also show how to apply ICP efficiently integrated with our method for causal discovery.
arXiv Detail & Related papers (2023-04-05T08:15:57Z) - VI-DGP: A variational inference method with deep generative prior for
solving high-dimensional inverse problems [0.7734726150561089]
We propose a novel approximation method for estimating the high-dimensional posterior distribution.
This approach leverages a deep generative model to learn a prior model capable of generating spatially-varying parameters.
The proposed method can be fully implemented in an automatic differentiation manner.
arXiv Detail & Related papers (2023-02-22T06:48:10Z) - On Representations of Mean-Field Variational Inference [2.4316550366482357]
We present a framework to analyze mean field variational inference (MFVI) algorithms.
Our approach enables the MFVI problem to be represented in three different manners.
Rigorous guarantees are established to show that a time-discretized implementation of the coordinate ascent variational inference algorithm yields a gradient flow in the limit.
arXiv Detail & Related papers (2022-10-20T16:26:22Z) - Error-Aware Spatial Ensembles for Video Frame Interpolation [50.63021118973639]
Video frame(VFI) algorithms have improved considerably in recent years due to unprecedented progress in both data-driven algorithms and their implementations.
Recent research has introduced advanced motion estimation or novel warping methods as the means to address challenging VFI scenarios.
This work introduces such a solution. By closely examining the correlation between optical flow and IE, the paper proposes novel error prediction metrics that partition the middle frame into distinct regions corresponding to different IE levels.
arXiv Detail & Related papers (2022-07-25T16:15:38Z) - Inverse Reinforcement Learning with Explicit Policy Estimates [19.159290496678004]
Various methods for solving the inverse reinforcement learning problem have been developed independently in machine learning and economics.
We show that they all belong to a class of optimization problems, characterized by a common form of gradient, the associated policy and the objective.
Using insights which emerge from our study of this class of optimization problems, we identify various problem scenarios and investigate each method's suitability for these problems.
arXiv Detail & Related papers (2021-03-04T07:00:58Z) - Semi-Supervised Learning with Variational Bayesian Inference and Maximum
Uncertainty Regularization [62.21716612888669]
We propose two generic methods for improving semi-supervised learning (SSL)
The first integrates weight perturbation (WP) into existing "consistency regularization" (CR) based methods.
The second method proposes a novel consistency loss called "maximum uncertainty regularization" (MUR)
arXiv Detail & Related papers (2020-12-03T09:49:35Z) - Learning explanations that are hard to vary [75.30552491694066]
We show that averaging across examples can favor memorization and patchwork' solutions that sew together different strategies.
We then propose and experimentally validate a simple alternative algorithm based on a logical AND.
arXiv Detail & Related papers (2020-09-01T10:17:48Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.