Recursive Bayesian Networks: Generalising and Unifying Probabilistic
Context-Free Grammars and Dynamic Bayesian Networks
- URL: http://arxiv.org/abs/2111.01853v1
- Date: Tue, 2 Nov 2021 19:21:15 GMT
- Title: Recursive Bayesian Networks: Generalising and Unifying Probabilistic
Context-Free Grammars and Dynamic Bayesian Networks
- Authors: Robert Lieck, Martin Rohrmeier
- Abstract summary: Probabilistic context-free grammars (PCFGs) and dynamic Bayesian networks (DBNs) are widely used sequence models with complementary strengths and limitations.
We present Recursive Bayesian Networks (RBNs), which generalise and unify PCFGs and DBNs, combining their strengths and containing both as special cases.
- Score: 0.0
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Probabilistic context-free grammars (PCFGs) and dynamic Bayesian networks
(DBNs) are widely used sequence models with complementary strengths and
limitations. While PCFGs allow for nested hierarchical dependencies (tree
structures), their latent variables (non-terminal symbols) have to be discrete.
In contrast, DBNs allow for continuous latent variables, but the dependencies
are strictly sequential (chain structure). Therefore, neither can be applied if
the latent variables are assumed to be continuous and also to have a nested
hierarchical dependency structure. In this paper, we present Recursive Bayesian
Networks (RBNs), which generalise and unify PCFGs and DBNs, combining their
strengths and containing both as special cases. RBNs define a joint
distribution over tree-structured Bayesian networks with discrete or continuous
latent variables. The main challenge lies in performing joint inference over
the exponential number of possible structures and the continuous variables. We
provide two solutions: 1) For arbitrary RBNs, we generalise inside and outside
probabilities from PCFGs to the mixed discrete-continuous case, which allows
for maximum posterior estimates of the continuous latent variables via gradient
descent, while marginalising over network structures. 2) For Gaussian RBNs, we
additionally derive an analytic approximation, allowing for robust parameter
optimisation and Bayesian inference. The capacity and diverse applications of
RBNs are illustrated on two examples: In a quantitative evaluation on synthetic
data, we demonstrate and discuss the advantage of RBNs for segmentation and
tree induction from noisy sequences, compared to change point detection and
hierarchical clustering. In an application to musical data, we approach the
unsolved problem of hierarchical music analysis from the raw note level and
compare our results to expert annotations.
Related papers
- Approximate learning of parsimonious Bayesian context trees [0.0]
The proposed framework is tested on synthetic and real-world data examples.
It outperforms existing sequence models when fitted to real protein sequences and honeypot computer terminal sessions.
arXiv Detail & Related papers (2024-07-27T11:50:40Z) - Joint Bayesian Inference of Graphical Structure and Parameters with a
Single Generative Flow Network [59.79008107609297]
We propose in this paper to approximate the joint posterior over the structure of a Bayesian Network.
We use a single GFlowNet whose sampling policy follows a two-phase process.
Since the parameters are included in the posterior distribution, this leaves more flexibility for the local probability models.
arXiv Detail & Related papers (2023-05-30T19:16:44Z) - GFlowNet-EM for learning compositional latent variable models [115.96660869630227]
A key tradeoff in modeling the posteriors over latents is between expressivity and tractable optimization.
We propose the use of GFlowNets, algorithms for sampling from an unnormalized density.
By training GFlowNets to sample from the posterior over latents, we take advantage of their strengths as amortized variational algorithms.
arXiv Detail & Related papers (2023-02-13T18:24:21Z) - DynGFN: Towards Bayesian Inference of Gene Regulatory Networks with
GFlowNets [81.75973217676986]
Gene regulatory networks (GRN) describe interactions between genes and their products that control gene expression and cellular function.
Existing methods either focus on challenge (1), identifying cyclic structure from dynamics, or on challenge (2) learning complex Bayesian posteriors over DAGs, but not both.
In this paper we leverage the fact that it is possible to estimate the "velocity" of gene expression with RNA velocity techniques to develop an approach that addresses both challenges.
arXiv Detail & Related papers (2023-02-08T16:36:40Z) - On the Foundations of Cycles in Bayesian Networks [4.312746668772342]
We present a foundational study regarding semantics for cyclic BNs that are generic and conservatively extend the cycle-free setting.
First, we propose constraint-based semantics that specify requirements for full joint distributions over a BN to be consistent with the local conditional probabilities and independencies.
Second, two kinds of limit semantics that formalize infinite unfolding approaches are introduced and shown to be computable by a Markov chain construction.
arXiv Detail & Related papers (2023-01-20T14:40:17Z) - Equivariant Transduction through Invariant Alignment [71.45263447328374]
We introduce a novel group-equivariant architecture that incorporates a group-in hard alignment mechanism.
We find that our network's structure allows it to develop stronger equivariant properties than existing group-equivariant approaches.
We additionally find that it outperforms previous group-equivariant networks empirically on the SCAN task.
arXiv Detail & Related papers (2022-09-22T11:19:45Z) - Bayesian Structure Learning with Generative Flow Networks [85.84396514570373]
In Bayesian structure learning, we are interested in inferring a distribution over the directed acyclic graph (DAG) from data.
Recently, a class of probabilistic models, called Generative Flow Networks (GFlowNets), have been introduced as a general framework for generative modeling.
We show that our approach, called DAG-GFlowNet, provides an accurate approximation of the posterior over DAGs.
arXiv Detail & Related papers (2022-02-28T15:53:10Z) - DiBS: Differentiable Bayesian Structure Learning [38.01659425023988]
We propose a general, fully differentiable framework for Bayesian structure learning (DiBS)
DiBS operates in the continuous space of a latent probabilistic graph representation.
Contrary to existing work, DiBS is agnostic to the form of the local conditional distributions.
arXiv Detail & Related papers (2021-05-25T11:23:08Z) - Unsupervised tree boosting for learning probability distributions [2.8444868155827634]
unsupervised tree boosting algorithm based on fitting additive tree ensembles.
Integral to the algorithm is a new notion of "residualization", i.e., subtracting a probability distribution from an observation to remove the distributional structure from the sampling distribution of the latter.
arXiv Detail & Related papers (2021-01-26T21:03:27Z) - A Constraint-Based Algorithm for the Structural Learning of
Continuous-Time Bayesian Networks [70.88503833248159]
We propose the first constraint-based algorithm for learning the structure of continuous-time Bayesian networks.
We discuss the different statistical tests and the underlying hypotheses used by our proposal to establish conditional independence.
arXiv Detail & Related papers (2020-07-07T07:34:09Z) - Consistency of Spectral Clustering on Hierarchical Stochastic Block
Models [5.983753938303726]
We study the hierarchy of communities in real-world networks under a generic block model.
We prove the strong consistency of this method under a wide range of model parameters.
Unlike most of existing work, our theory covers multiscale networks where the connection probabilities may differ by orders of magnitude.
arXiv Detail & Related papers (2020-04-30T01:08:59Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.