The Quantified Boolean Bayesian Network: Theory and Experiments with a Logical Graphical Model

• URL: http://arxiv.org/abs/2402.06557v1
• Date: Fri, 9 Feb 2024 17:15:45 GMT
• Title: The Quantified Boolean Bayesian Network: Theory and Experiments with a Logical Graphical Model
• Authors: Gregory Coppola
• Abstract summary: We show how a probabilistic Network can be configured to represent the logical reasoning underlying human language. For inference, we investigate the use of Loopy Belief Propagation (LBP), which is not guaranteed to converge. Our experiments show that LBP indeed does converge very reliably, and our analysis shows that a round of LBP takes time $O(N2n)$, where $N$ bounds the number of variables considered.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper introduces the Quantified Boolean Bayesian Network (QBBN), which provides a unified view of logical and probabilistic reasoning. The QBBN is meant to address a central problem with the Large Language Model (LLM), which has become extremely popular in Information Retrieval, which is that the LLM hallucinates. A Bayesian Network, by construction, cannot hallucinate, because it can only return answers that it can explain. We show how a Bayesian Network over an unbounded number of boolean variables can be configured to represent the logical reasoning underlying human language. We do this by creating a key-value version of the First-Order Calculus, for which we can prove consistency and completeness. We show that the model is trivially trained over fully observed data, but that inference is non-trivial. Exact inference in a Bayesian Network is intractable (i.e. $\Omega(2^N)$ for $N$ variables). For inference, we investigate the use of Loopy Belief Propagation (LBP), which is not guaranteed to converge, but which has been shown to often converge in practice. Our experiments show that LBP indeed does converge very reliably, and our analysis shows that a round of LBP takes time $O(N2^n)$, where $N$ bounds the number of variables considered, and $n$ bounds the number of incoming connections to any factor, and further improvements may be possible. Our network is specifically designed to alternate between AND and OR gates in a Boolean Algebra, which connects more closely to logical reasoning, allowing a completeness proof for an expanded version of our network, and also allows inference to follow specific but adequate pathways, that turn out to be fast.

Related papers

• Bayesian Inference with Deep Weakly Nonlinear Networks [57.95116787699412]
  We show at a physics level of rigor that Bayesian inference with a fully connected neural network is solvable. We provide techniques to compute the model evidence and posterior to arbitrary order in $1/N$ and at arbitrary temperature.
  arXiv  Detail & Related papers  (2024-05-26T17:08:04Z)
• LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Provers [60.009969929857704]
  Logical reasoning is an important task for artificial intelligence with potential impacts on science, mathematics, and society. In this work, we reformulating such tasks as modular neurosymbolic programming, which we call LINC. We observe significant performance gains on FOLIO and a balanced subset of ProofWriter for three different models in nearly all experimental conditions we evaluate.
  arXiv  Detail & Related papers  (2023-10-23T17:58:40Z)
• Neural Network Field Theories: Non-Gaussianity, Actions, and Locality [0.0]
  Both the path integral measure in field theory and ensembles of neural networks describe distributions over functions. An expansion in $1/N$ corresponds to interactions in the field theory, but others, such as in a small breaking of the statistical independence of network parameters, can also lead to interacting theories.
  arXiv  Detail & Related papers  (2023-07-06T18:00:01Z)
• Probabilistic Modeling: Proving the Lottery Ticket Hypothesis in Spiking Neural Network [30.924449325020767]
  Lottery Ticket Hypothesis (LTH) states that a randomly-d large neural network contains a small sub-network. LTH opens up a new path for pruning network.
  arXiv  Detail & Related papers  (2023-05-20T09:27:34Z)
• p-Adic Statistical Field Theory and Deep Belief Networks [0.0]
  In general quantum field theories over a $p$-adic spacetime can be formulated in a rigorous way. We show these theories are deeply connected with the deep belief networks (DBNs) In our approach, a $p$-adic SFT corresponds to a $p$-adic continuous DBN, and a discretization of this theory corresponds to a $p$-adic discrete DBN.
  arXiv  Detail & Related papers  (2022-07-28T04:04:48Z)
• Dual Lottery Ticket Hypothesis [71.95937879869334]
  Lottery Ticket Hypothesis (LTH) provides a novel view to investigate sparse network training and maintain its capacity. In this work, we regard the winning ticket from LTH as the subnetwork which is in trainable condition and its performance as our benchmark. We propose a simple sparse network training strategy, Random Sparse Network Transformation (RST), to substantiate our DLTH.
  arXiv  Detail & Related papers  (2022-03-08T18:06:26Z)
• On the Optimal Memorization Power of ReLU Neural Networks [53.15475693468925]
  We show that feedforward ReLU neural networks can memorization any $N$ points that satisfy a mild separability assumption. We prove that having such a large bit complexity is both necessary and sufficient for memorization with a sub-linear number of parameters.
  arXiv  Detail & Related papers  (2021-10-07T05:25:23Z)
• Abstraction, Reasoning and Deep Learning: A Study of the "Look and Say" Sequence [0.0]
  Deep neural networks can exhibit high competence' (as measured by accuracy) when trained on large data sets. We report on two sets experiments on the Look and Say" puzzle data. Despite the amazing accuracy (on both, training and test data), the performance of the trained programs on the actual L&S sequence is bad.
  arXiv  Detail & Related papers  (2021-09-27T01:41:37Z)
• A simple geometric proof for the benefit of depth in ReLU networks [57.815699322370826]
  We present a simple proof for the benefit of depth in multi-layer feedforward network with rectified activation ("depth separation") We present a concrete neural network with linear depth (in $m$) and small constant width ($leq 4$) that classifies the problem with zero error.
  arXiv  Detail & Related papers  (2021-01-18T15:40:27Z)
• Bayesian Inference Forgetting [82.6681466124663]
  The right to be forgotten has been legislated in many countries but the enforcement in machine learning would cause unbearable costs. This paper proposes a it Bayesian inference forgetting (BIF) framework to realize the right to be forgotten in Bayesian inference.
  arXiv  Detail & Related papers  (2021-01-16T09:52:51Z)
• Optimal Lottery Tickets via SubsetSum: Logarithmic Over-Parameterization is Sufficient [9.309655246559094]
  We show that any target network of width $d$ and depth $l$ can be approximated by pruning a random network that is a factor $O(log(dl))$ wider and twice as deep. Our analysis relies on connecting pruning random ReLU networks to random instances of the textscSubset problem.
  arXiv  Detail & Related papers  (2020-06-14T19:32:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.