Bayesian Structure Scores for Probabilistic Circuits

• URL: http://arxiv.org/abs/2302.12130v1
• Date: Thu, 23 Feb 2023 16:12:19 GMT
• Title: Bayesian Structure Scores for Probabilistic Circuits
• Authors: Yang Yang, Gennaro Gala and Robert Peharz
• Abstract summary: Probabilistic circuits (PCs) are a prominent representation of probability with tractable inference. We develop Bayesian structure scores for deterministic PCs, i.e., structure likelihood with parameters marginalized out. We achieve good trade-offs between training time and model fit in terms of log-likelihood.
• Score: 13.441379161477272
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Probabilistic circuits (PCs) are a prominent representation of probability distributions with tractable inference. While parameter learning in PCs is rigorously studied, structure learning is often more based on heuristics than on principled objectives. In this paper, we develop Bayesian structure scores for deterministic PCs, i.e., the structure likelihood with parameters marginalized out, which are well known as rigorous objectives for structure learning in probabilistic graphical models. When used within a greedy cutset algorithm, our scores effectively protect against overfitting and yield a fast and almost hyper-parameter-free structure learner, distinguishing it from previous approaches. In experiments, we achieve good trade-offs between training time and model fit in terms of log-likelihood. Moreover, the principled nature of Bayesian scores unlocks PCs for accommodating frameworks such as structural expectation-maximization.

Related papers

• Conformal Predictions for Probabilistically Robust Scalable Machine Learning Classification [1.757077789361314]
  Conformal predictions make it possible to define reliable and robust learning algorithms. They are essentially a method for evaluating whether an algorithm is good enough to be used in practice. This paper defines a reliable learning framework for classification from the very beginning of its design.
  arXiv  Detail & Related papers  (2024-03-15T14:59:24Z)
• Probabilistic Dataset Reconstruction from Interpretable Models [8.31111379034875]
  We show that optimal interpretable models are often more compact and leak less information regarding their training data than greedily-built ones. Our results suggest that optimal interpretable models are often more compact and leak less information regarding their training data than greedily-built ones.
  arXiv  Detail & Related papers  (2023-08-29T08:10:09Z)
• Provable Guarantees for Generative Behavior Cloning: Bridging Low-Level Stability and High-Level Behavior [51.60683890503293]
  We propose a theoretical framework for studying behavior cloning of complex expert demonstrations using generative modeling. We show that pure supervised cloning can generate trajectories matching the per-time step distribution of arbitrary expert trajectories.
  arXiv  Detail & Related papers  (2023-07-27T04:27:26Z)
• When Does Confidence-Based Cascade Deferral Suffice? [69.28314307469381]
  Cascades are a classical strategy to enable inference cost to vary adaptively across samples. A deferral rule determines whether to invoke the next classifier in the sequence, or to terminate prediction. Despite being oblivious to the structure of the cascade, confidence-based deferral often works remarkably well in practice.
  arXiv  Detail & Related papers  (2023-07-06T04:13:57Z)
• Compositional Probabilistic and Causal Inference using Tractable Circuit Models [20.07977560803858]
  We introduce md-vtrees, a novel structural formulation of (marginal) determinism in structured decomposable PCs. We derive the first polytime algorithms for causal inference queries such as backdoor adjustment on PCs.
  arXiv  Detail & Related papers  (2023-04-17T13:48:16Z)
• Structural Learning of Probabilistic Sentential Decision Diagrams under Partial Closed-World Assumption [127.439030701253]
  Probabilistic sentential decision diagrams are a class of structured-decomposable circuits. We propose a new scheme based on a partial closed-world assumption: data implicitly provide the logical base of the circuit. Preliminary experiments show that the proposed approach might properly fit training data, and generalize well to test data, provided that these remain consistent with the underlying logical base.
  arXiv  Detail & Related papers  (2021-07-26T12:01:56Z)
• Probabilistic Generating Circuits [50.98473654244851]
  We propose probabilistic generating circuits (PGCs) for their efficient representation. PGCs are not just a theoretical framework that unifies vastly different existing models, but also show huge potential in modeling realistic data. We exhibit a simple class of PGCs that are not trivially subsumed by simple combinations of PCs and DPPs, and obtain competitive performance on a suite of density estimation benchmarks.
  arXiv  Detail & Related papers  (2021-02-19T07:06:53Z)
• Differentiable TAN Structure Learning for Bayesian Network Classifiers [19.30562170076368]
  We consider learning of tree-augmented naive Bayes (TAN) structures for Bayesian network classifiers with discrete input features. Instead of performing a optimization over the space of possible graph structures, the proposed method learns a distribution over graph structures. Our method consistently outperforms random TAN structures and Chow-Liu TAN structures.
  arXiv  Detail & Related papers  (2020-08-21T16:22:47Z)
• Strudel: Learning Structured-Decomposable Probabilistic Circuits [37.153542210716004]
  Strudel is a simple, fast and accurate learning algorithm for structured-decomposable PCs. It delivers more accurate single PC models in fewer iterations, and dramatically scales learning when building ensembles of PCs. We show these advantages on standard density estimation benchmarks and challenging inference scenarios.
  arXiv  Detail & Related papers  (2020-07-18T04:51:31Z)
• A Constraint-Based Algorithm for the Structural Learning of Continuous-Time Bayesian Networks [70.88503833248159]
  We propose the first constraint-based algorithm for learning the structure of continuous-time Bayesian networks. We discuss the different statistical tests and the underlying hypotheses used by our proposal to establish conditional independence.
  arXiv  Detail & Related papers  (2020-07-07T07:34:09Z)
• Predictive Coding Approximates Backprop along Arbitrary Computation Graphs [68.8204255655161]
  We develop a strategy to translate core machine learning architectures into their predictive coding equivalents. Our models perform equivalently to backprop on challenging machine learning benchmarks. Our method raises the potential that standard machine learning algorithms could in principle be directly implemented in neural circuitry.
  arXiv  Detail & Related papers  (2020-06-07T15:35:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.