Scaling Continuous Latent Variable Models as Probabilistic Integral Circuits

• URL: http://arxiv.org/abs/2406.06494v1
• Date: Mon, 10 Jun 2024 17:30:17 GMT
• Title: Scaling Continuous Latent Variable Models as Probabilistic Integral Circuits
• Authors: Gennaro Gala, Cassio de Campos, Antonio Vergari, Erik Quaeghebeur,
• Abstract summary: Probabilistic integral circuits (PICs) are symbolic computational graphs defining continuous latent variables (LVs) PICs are tractable if LVs can be analytically integrated out, otherwise they can be approximated by tractable probabilistic circuits (PC) We present a pipeline for building DAG-shaped PICs out of arbitrary variable decompositions, a procedure for training PICs using tensorized circuit architectures, and neural functional sharing techniques.
• Score: 5.969243233796684
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Probabilistic integral circuits (PICs) have been recently introduced as probabilistic models enjoying the key ingredient behind expressive generative models: continuous latent variables (LVs). PICs are symbolic computational graphs defining continuous LV models as hierarchies of functions that are summed and multiplied together, or integrated over some LVs. They are tractable if LVs can be analytically integrated out, otherwise they can be approximated by tractable probabilistic circuits (PC) encoding a hierarchical numerical quadrature process, called QPCs. So far, only tree-shaped PICs have been explored, and training them via numerical quadrature requires memory-intensive processing at scale. In this paper, we address these issues, and present: (i) a pipeline for building DAG-shaped PICs out of arbitrary variable decompositions, (ii) a procedure for training PICs using tensorized circuit architectures, and (iii) neural functional sharing techniques to allow scalable training. In extensive experiments, we showcase the effectiveness of functional sharing and the superiority of QPCs over traditional PCs.

Related papers

• Memory-Augmented Hybrid Quantum Reservoir Computing [0.0]
  We present a hybrid quantum-classical approach that implements memory through classical post-processing of quantum measurements. We tested our model on two physical platforms: a fully connected Ising model and a Rydberg atom array.
  arXiv  Detail & Related papers  (2024-09-15T22:44:09Z)
• Probabilistic Integral Circuits [11.112802758446344]
  We introduce a new language of computational graphs that extends PCs with integral units representing continuous LVs. In practice, we parameterise PICs with light-weight neural nets delivering an intractable hierarchical continuous mixture. We show that such PIC-approximating PCs systematically outperform PCs commonly learned via expectation-maximization or SGD.
  arXiv  Detail & Related papers  (2023-10-25T20:38:18Z)
• Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection [88.23337313766353]
  This work first provides a comprehensive statistical theory for transformers to perform ICL. We show that transformers can implement a broad class of standard machine learning algorithms in context. A emphsingle transformer can adaptively select different base ICL algorithms.
  arXiv  Detail & Related papers  (2023-06-07T17:59:31Z)
• Pretraining Graph Neural Networks for few-shot Analog Circuit Modeling and Design [68.1682448368636]
  We present a supervised pretraining approach to learn circuit representations that can be adapted to new unseen topologies or unseen prediction tasks. To cope with the variable topological structure of different circuits we describe each circuit as a graph and use graph neural networks (GNNs) to learn node embeddings. We show that pretraining GNNs on prediction of output node voltages can encourage learning representations that can be adapted to new unseen topologies or prediction of new circuit level properties.
  arXiv  Detail & Related papers  (2022-03-29T21:18:47Z)
• Protocols for Trainable and Differentiable Quantum Generative Modelling [21.24186888129542]
  We propose an approach for learning probability distributions as differentiable quantum circuits (DQC) We perform training of a DQC-based model, where data is encoded in a latent space with a phase feature map, followed by a variational quantum circuit. This allows fast sampling from parametrized distributions using a single-shot readout.
  arXiv  Detail & Related papers  (2022-02-16T18:55:48Z)
• Probabilistic Generating Circuits [50.98473654244851]
  We propose probabilistic generating circuits (PGCs) for their efficient representation. PGCs are not just a theoretical framework that unifies vastly different existing models, but also show huge potential in modeling realistic data. We exhibit a simple class of PGCs that are not trivially subsumed by simple combinations of PCs and DPPs, and obtain competitive performance on a suite of density estimation benchmarks.
  arXiv  Detail & Related papers  (2021-02-19T07:06:53Z)
• Characterizing the loss landscape of variational quantum circuits [77.34726150561087]
  We introduce a way to compute the Hessian of the loss function of VQCs. We show how this information can be interpreted and compared to classical neural networks.
  arXiv  Detail & Related papers  (2020-08-06T17:48:12Z)
• Predictive Coding Approximates Backprop along Arbitrary Computation Graphs [68.8204255655161]
  We develop a strategy to translate core machine learning architectures into their predictive coding equivalents. Our models perform equivalently to backprop on challenging machine learning benchmarks. Our method raises the potential that standard machine learning algorithms could in principle be directly implemented in neural circuitry.
  arXiv  Detail & Related papers  (2020-06-07T15:35:47Z)
• Einsum Networks: Fast and Scalable Learning of Tractable Probabilistic Circuits [99.59941892183454]
  We propose Einsum Networks (EiNets), a novel implementation design for PCs. At their core, EiNets combine a large number of arithmetic operations in a single monolithic einsum-operation. We show that the implementation of Expectation-Maximization (EM) can be simplified for PCs, by leveraging automatic differentiation.
  arXiv  Detail & Related papers  (2020-04-13T23:09:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.