Probabilistic Neural Circuits
- URL: http://arxiv.org/abs/2403.06235v1
- Date: Sun, 10 Mar 2024 15:25:49 GMT
- Title: Probabilistic Neural Circuits
- Authors: Pedro Zuidberg Dos Martires
- Abstract summary: Probabilistic neural circuits (PNCs) strike a balance between PCs and neural nets in terms of tractability and expressive power.
We show that PNCs can be interpreted as deep mixtures of Bayesian networks.
- Score: 4.724177741282789
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Probabilistic circuits (PCs) have gained prominence in recent years as a
versatile framework for discussing probabilistic models that support tractable
queries and are yet expressive enough to model complex probability
distributions. Nevertheless, tractability comes at a cost: PCs are less
expressive than neural networks. In this paper we introduce probabilistic
neural circuits (PNCs), which strike a balance between PCs and neural nets in
terms of tractability and expressive power. Theoretically, we show that PNCs
can be interpreted as deep mixtures of Bayesian networks. Experimentally, we
demonstrate that PNCs constitute powerful function approximators.
Related papers
- Contrastive Learning in Memristor-based Neuromorphic Systems [55.11642177631929]
Spiking neural networks have become an important family of neuron-based models that sidestep many of the key limitations facing modern-day backpropagation-trained deep networks.
In this work, we design and investigate a proof-of-concept instantiation of contrastive-signal-dependent plasticity (CSDP), a neuromorphic form of forward-forward-based, backpropagation-free learning.
arXiv Detail & Related papers (2024-09-17T04:48:45Z) - Scalable Neural-Probabilistic Answer Set Programming [18.136093815001423]
We introduce SLASH, a novel DPPL that consists of Neural-Probabilistic Predicates (NPPs) and a logic program, united via answer set programming (ASP)
We show how to prune the insignificantally insignificant parts of the (ground) program, speeding up reasoning without sacrificing the predictive performance.
We evaluate SLASH on a variety of different tasks, including the benchmark task of MNIST addition and Visual Question Answering (VQA)
arXiv Detail & Related papers (2023-06-14T09:45:29Z) - Tractable Probabilistic Graph Representation Learning with Graph-Induced
Sum-Product Networks [25.132159381873656]
We introduce Graph-Induced Sum-Product Networks (GSPNs), a new probabilistic framework for graph representation learning.
We show the model's competitiveness on scarce supervision scenarios, under missing data, and for graph classification in comparison to popular neural models.
arXiv Detail & Related papers (2023-05-17T20:02:08Z) - Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a
Polynomial Net Study [55.12108376616355]
The study on NTK has been devoted to typical neural network architectures, but is incomplete for neural networks with Hadamard products (NNs-Hp)
In this work, we derive the finite-width-K formulation for a special class of NNs-Hp, i.e., neural networks.
We prove their equivalence to the kernel regression predictor with the associated NTK, which expands the application scope of NTK.
arXiv Detail & Related papers (2022-09-16T06:36:06Z) - ExSpliNet: An interpretable and expressive spline-based neural network [0.3867363075280544]
We present ExSpliNet, an interpretable and expressive neural network model.
We give a probabilistic interpretation of the model and show its universal approximation properties.
arXiv Detail & Related papers (2022-05-03T14:06:36Z) - The Spectral Bias of Polynomial Neural Networks [63.27903166253743]
Polynomial neural networks (PNNs) have been shown to be particularly effective at image generation and face recognition, where high-frequency information is critical.
Previous studies have revealed that neural networks demonstrate a $textitspectral bias$ towards low-frequency functions, which yields faster learning of low-frequency components during training.
Inspired by such studies, we conduct a spectral analysis of the Tangent Kernel (NTK) of PNNs.
We find that the $Pi$-Net family, i.e., a recently proposed parametrization of PNNs, speeds up the
arXiv Detail & Related papers (2022-02-27T23:12:43Z) - HyperSPNs: Compact and Expressive Probabilistic Circuits [89.897635970366]
HyperSPNs is a new paradigm of generating the mixture weights of large PCs using a small-scale neural network.
We show the merits of our regularization strategy on two state-of-the-art PC families introduced in recent literature.
arXiv Detail & Related papers (2021-12-02T01:24:43Z) - PAC-Bayesian Learning of Aggregated Binary Activated Neural Networks
with Probabilities over Representations [2.047424180164312]
We study the expectation of a probabilistic neural network as a predictor by itself, focusing on the aggregation of binary activated neural networks with normal distributions over real-valued weights.
We show that the exact computation remains tractable for deep but narrow neural networks, thanks to a dynamic programming approach.
arXiv Detail & Related papers (2021-10-28T14:11:07Z) - Mitigating Performance Saturation in Neural Marked Point Processes:
Architectures and Loss Functions [50.674773358075015]
We propose a simple graph-based network structure called GCHP, which utilizes only graph convolutional layers.
We show that GCHP can significantly reduce training time and the likelihood ratio loss with interarrival time probability assumptions can greatly improve the model performance.
arXiv Detail & Related papers (2021-07-07T16:59:14Z) - Probabilistic Deep Learning with Probabilistic Neural Networks and Deep
Probabilistic Models [0.6091702876917281]
We distinguish two approaches to probabilistic deep learning: probabilistic neural networks and deep probabilistic models.
Probabilistic deep learning is deep learning that accounts for uncertainty, both model uncertainty and data uncertainty.
arXiv Detail & Related papers (2021-05-31T22:13:21Z) - Probabilistic Generating Circuits [50.98473654244851]
We propose probabilistic generating circuits (PGCs) for their efficient representation.
PGCs are not just a theoretical framework that unifies vastly different existing models, but also show huge potential in modeling realistic data.
We exhibit a simple class of PGCs that are not trivially subsumed by simple combinations of PCs and DPPs, and obtain competitive performance on a suite of density estimation benchmarks.
arXiv Detail & Related papers (2021-02-19T07:06:53Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.