Probabilistic Graphical Models and Tensor Networks: A Hybrid Framework

• URL: http://arxiv.org/abs/2106.15666v1
• Date: Tue, 29 Jun 2021 18:20:44 GMT
• Title: Probabilistic Graphical Models and Tensor Networks: A Hybrid Framework
• Abstract summary: We introduce a hybrid PGM-TN formalism that integrates quantum-like correlations into PGM models in a principled manner. We experimentally verify the performance of such hybrid models in a sequential modeling task.
• Score: 3.9533044769534444
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We investigate a correspondence between two formalisms for discrete probabilistic modeling: probabilistic graphical models (PGMs) and tensor networks (TNs), a powerful modeling framework for simulating complex quantum systems. The graphical calculus of PGMs and TNs exhibits many similarities, with discrete undirected graphical models (UGMs) being a special case of TNs. However, more general probabilistic TN models such as Born machines (BMs) employ complex-valued hidden states to produce novel forms of correlation among the probabilities. While representing a new modeling resource for capturing structure in discrete probability distributions, this behavior also renders the direct application of standard PGM tools impossible. We aim to bridge this gap by introducing a hybrid PGM-TN formalism that integrates quantum-like correlations into PGM models in a principled manner, using the physically-motivated concept of decoherence. We first prove that applying decoherence to the entirety of a BM model converts it into a discrete UGM, and conversely, that any subgraph of a discrete UGM can be represented as a decohered BM. This method allows a broad family of probabilistic TN models to be encoded as partially decohered BMs, a fact we leverage to combine the representational strengths of both model families. We experimentally verify the performance of such hybrid models in a sequential modeling task, and identify promising uses of our method within the context of existing applications of graphical models.

Related papers

• A Censored Transformed Model for Proportional Outcomes with Boundary Mass and an Application to Loss Given Default Modeling [6.789370732159177]
  We introduce the zero-one censored transformed normal (ZOCTN) model for proportional responses with potential probability mass at the boundaries 0 and 1.<n>We characterize the transformation parameters establish large-sample properties, and relate the affine-logit specification to broader classes of interior distributions.
  arXiv  Detail & Related papers  (2026-06-19T15:10:41Z)
• Guiding Time-Varying Generative Models with Natural Gradients on Exponential Family Manifold [5.000311680307273]
  We show that the evolution of time-varying generative models can be projected onto an exponential family manifold. We then train the generative model by moving its projection on the manifold according to the natural gradient descent scheme. We propose particle versions of the algorithm, which feature closed-form update rules for any parametric model within the exponential family.
  arXiv  Detail & Related papers  (2025-02-11T15:39:47Z)
• [MASK] is All You Need [28.90875822599164]
  We propose using discrete-state models to connect Masked Generative and Non-autoregressive Diffusion models. By leveraging [MASK] in discrete-state models, we can bridge Masked Generative and Non-autoregressive Diffusion models.
  arXiv  Detail & Related papers  (2024-12-09T18:59:56Z)
• Likelihood Based Inference in Fully and Partially Observed Exponential Family Graphical Models with Intractable Normalizing Constants [4.532043501030714]
  Probabilistic graphical models that encode an underlying Markov random field are fundamental building blocks of generative modeling. This paper is to demonstrate that full likelihood based analysis of these models is feasible in a computationally efficient manner.
  arXiv  Detail & Related papers  (2024-04-27T02:58:22Z)
• Wasserstein proximal operators describe score-based generative models and resolve memorization [12.321631823103894]
  We first formulate SGMs in terms of the Wasserstein proximal operator (WPO) We show that WPO describes the inductive bias of diffusion and score-based models. We present an interpretable kernel-based model for the score function which dramatically improves the performance of SGMs.
  arXiv  Detail & Related papers  (2024-02-09T03:33:13Z)
• Low-Rank Constraints for Fast Inference in Structured Models [110.38427965904266]
  This work demonstrates a simple approach to reduce the computational and memory complexity of a large class of structured models. Experiments with neural parameterized structured models for language modeling, polyphonic music modeling, unsupervised grammar induction, and video modeling show that our approach matches the accuracy of standard models at large state spaces.
  arXiv  Detail & Related papers  (2022-01-08T00:47:50Z)
• PSD Representations for Effective Probability Models [117.35298398434628]
  We show that a recently proposed class of positive semi-definite (PSD) models for non-negative functions is particularly suited to this end. We characterize both approximation and generalization capabilities of PSD models, showing that they enjoy strong theoretical guarantees. Our results open the way to applications of PSD models to density estimation, decision theory and inference.
  arXiv  Detail & Related papers  (2021-06-30T15:13:39Z)
• Relational VAE: A Continuous Latent Variable Model for Graph Structured Data [0.0]
  We show applications on the problem of structured probability density modeling for simulated and real wind farm monitoring data. We release the source code, along with the simulated datasets.
  arXiv  Detail & Related papers  (2021-06-30T13:24:27Z)
• Continual Learning with Fully Probabilistic Models [70.3497683558609]
  We present an approach for continual learning based on fully probabilistic (or generative) models of machine learning. We propose a pseudo-rehearsal approach using a Gaussian Mixture Model (GMM) instance for both generator and classifier functionalities. We show that GMR achieves state-of-the-art performance on common class-incremental learning problems at very competitive time and memory complexity.
  arXiv  Detail & Related papers  (2021-04-19T12:26:26Z)
• Probabilistic Generating Circuits [50.98473654244851]
  We propose probabilistic generating circuits (PGCs) for their efficient representation. PGCs are not just a theoretical framework that unifies vastly different existing models, but also show huge potential in modeling realistic data. We exhibit a simple class of PGCs that are not trivially subsumed by simple combinations of PCs and DPPs, and obtain competitive performance on a suite of density estimation benchmarks.
  arXiv  Detail & Related papers  (2021-02-19T07:06:53Z)
• Improving the Reconstruction of Disentangled Representation Learners via Multi-Stage Modeling [54.94763543386523]
  Current autoencoder-based disentangled representation learning methods achieve disentanglement by penalizing the ( aggregate) posterior to encourage statistical independence of the latent factors. We present a novel multi-stage modeling approach where the disentangled factors are first learned using a penalty-based disentangled representation learning method. Then, the low-quality reconstruction is improved with another deep generative model that is trained to model the missing correlated latent variables.
  arXiv  Detail & Related papers  (2020-10-25T18:51:15Z)
• Probabilistic Circuits for Variational Inference in Discrete Graphical Models [101.28528515775842]
  Inference in discrete graphical models with variational methods is difficult. Many sampling-based methods have been proposed for estimating Evidence Lower Bound (ELBO) We propose a new approach that leverages the tractability of probabilistic circuit models, such as Sum Product Networks (SPN) We show that selective-SPNs are suitable as an expressive variational distribution, and prove that when the log-density of the target model is aweighted the corresponding ELBO can be computed analytically.
  arXiv  Detail & Related papers  (2020-10-22T05:04:38Z)
• Quantum Tensor Networks, Stochastic Processes, and Weighted Automata [31.641069053835192]
  Similar models have been studied in the processes and weighted automata literature. We show how stationary or uniform versions of popular quantum tensor network models have equivalent representations in the processes and weighted automata literature.
  arXiv  Detail & Related papers  (2020-10-20T22:28:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.