Polynomial Semantics of Tractable Probabilistic Circuits

• URL: http://arxiv.org/abs/2402.09085v3
• Date: Thu, 8 Aug 2024 05:58:30 GMT
• Title: Polynomial Semantics of Tractable Probabilistic Circuits
• Authors: Oliver Broadrick, Honghua Zhang, Guy Van den Broeck,
• Abstract summary: We show that each of these circuit models is equivalent in the sense that any circuit for one of them can be transformed into a circuit for any of the others with only an increase in size. They are all tractable for marginal inference on the same class of distributions.
• Score: 29.3642918977097
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Probabilistic circuits compute multilinear polynomials that represent multivariate probability distributions. They are tractable models that support efficient marginal inference. However, various polynomial semantics have been considered in the literature (e.g., network polynomials, likelihood polynomials, generating functions, and Fourier transforms). The relationships between circuit representations of these polynomial encodings of distributions is largely unknown. In this paper, we prove that for distributions over binary variables, each of these probabilistic circuit models is equivalent in the sense that any circuit for one of them can be transformed into a circuit for any of the others with only a polynomial increase in size. They are therefore all tractable for marginal inference on the same class of distributions. Finally, we explore the natural extension of one such polynomial semantics, called probabilistic generating circuits, to categorical random variables, and establish that inference becomes #P-hard.

Related papers

• Conditional Distribution Quantization in Machine Learning [83.54039134248231]
  Conditional expectation mathbbE(Y mid X) often fails to capture the complexity of multimodal conditional distributions mathcalL(Y mid X) We propose using n-point conditional quantizations--functional mappings of X that are learnable via gradient descent--to approximate mathcalL(Y mid X)
  arXiv  Detail & Related papers  (2025-02-11T00:28:24Z)
• Polynomial Threshold Functions of Bounded Tree-Width: Some Explainability and Complexity Aspects [0.6554326244334868]
  The tree-width of a multivariate is the tree-width of the hypergraph with hyperedges corresponding to its terms. A representation of a Boolean function as the sign of a bounded tree-width is called a threshold representation.
  arXiv  Detail & Related papers  (2025-01-14T18:28:08Z)
• Probabilistic Circuits for Cumulative Distribution Functions [27.511363113215374]
  We show that for distributions over binary random variables these representations (PMF and CDF) are equivalent, in the sense that one can be transformed to the other in time. For continuous variables, smooth, decomposable PCs computing PDFs and CDFs can be efficiently transformed to each other by modifying only the leaves of the circuit.
  arXiv  Detail & Related papers  (2024-08-08T05:33:21Z)
• Tensor cumulants for statistical inference on invariant distributions [49.80012009682584]
  We show that PCA becomes computationally hard at a critical value of the signal's magnitude. We define a new set of objects, which provide an explicit, near-orthogonal basis for invariants of a given degree. It also lets us analyze a new problem of distinguishing between different ensembles.
  arXiv  Detail & Related papers  (2024-04-29T14:33:24Z)
• Matching Normalizing Flows and Probability Paths on Manifolds [57.95251557443005]
  Continuous Normalizing Flows (CNFs) are generative models that transform a prior distribution to a model distribution by solving an ordinary differential equation (ODE) We propose to train CNFs by minimizing probability path divergence (PPD), a novel family of divergences between the probability density path generated by the CNF and a target probability density path. We show that CNFs learned by minimizing PPD achieve state-of-the-art results in likelihoods and sample quality on existing low-dimensional manifold benchmarks.
  arXiv  Detail & Related papers  (2022-07-11T08:50:19Z)
• Polynomial decompositions with invariance and positivity inspired by tensors [1.433758865948252]
  This framework has been recently introduced for tensor decompositions, in particular for quantum many-body systems. We define invariant decompositions of structures, approximations, and undecidability to reals. Our work sheds new light on footings by putting them on an equal footing with tensors, and opens the door to extending this framework to other product structures.
  arXiv  Detail & Related papers  (2021-09-14T13:30:50Z)
• Probabilistic Generating Circuits [50.98473654244851]
  We propose probabilistic generating circuits (PGCs) for their efficient representation. PGCs are not just a theoretical framework that unifies vastly different existing models, but also show huge potential in modeling realistic data. We exhibit a simple class of PGCs that are not trivially subsumed by simple combinations of PCs and DPPs, and obtain competitive performance on a suite of density estimation benchmarks.
  arXiv  Detail & Related papers  (2021-02-19T07:06:53Z)
• Probabilistic Circuits for Variational Inference in Discrete Graphical Models [101.28528515775842]
  Inference in discrete graphical models with variational methods is difficult. Many sampling-based methods have been proposed for estimating Evidence Lower Bound (ELBO) We propose a new approach that leverages the tractability of probabilistic circuit models, such as Sum Product Networks (SPN) We show that selective-SPNs are suitable as an expressive variational distribution, and prove that when the log-density of the target model is aweighted the corresponding ELBO can be computed analytically.
  arXiv  Detail & Related papers  (2020-10-22T05:04:38Z)
• SURF: A Simple, Universal, Robust, Fast Distribution Learning Algorithm [64.13217062232874]
  SURF is an algorithm for approximating distributions by piecewises. It outperforms state-of-the-art algorithms in experiments.
  arXiv  Detail & Related papers  (2020-02-22T01:03:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.