Scalability of Bayesian Network Structure Elicitation with Large Language Models: a Novel Methodology and Comparative Analysis

• URL: http://arxiv.org/abs/2407.09311v1
• Date: Fri, 12 Jul 2024 14:52:13 GMT
• Title: Scalability of Bayesian Network Structure Elicitation with Large Language Models: a Novel Methodology and Comparative Analysis
• Authors: Nikolay Babakov, Ehud Reiter, Alberto Bugarin,
• Abstract summary: We propose a novel method for Bayesian Networks (BNs) structure elicitation based on several LLMs with different experiences. We compare the method with one alternative method on various widely and not widely known BNs of different sizes and study the scalability of both methods on them.
• Score: 5.91003502313675
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this work, we propose a novel method for Bayesian Networks (BNs) structure elicitation that is based on the initialization of several LLMs with different experiences, independently querying them to create a structure of the BN, and further obtaining the final structure by majority voting. We compare the method with one alternative method on various widely and not widely known BNs of different sizes and study the scalability of both methods on them. We also propose an approach to check the contamination of BNs in LLM, which shows that some widely known BNs are inapplicable for testing the LLM usage for BNs structure elicitation. We also show that some BNs may be inapplicable for such experiments because their node names are indistinguishable. The experiments on the other BNs show that our method performs better than the existing method with one of the three studied LLMs; however, the performance of both methods significantly decreases with the increase in BN size.

Related papers

• Scalable Ensemble Diversification for OOD Generalization and Detection [68.8982448081223]
  SED identifies hard training samples on the fly and encourages the ensemble members to disagree on these. We show how to avoid the expensive computations in existing methods of exhaustive pairwise disagreements across models. For OOD generalization, we observe large benefits from the diversification in multiple settings including output-space (classical) ensembles and weight-space ensembles (model soups)
  arXiv  Detail & Related papers  (2024-09-25T10:30:24Z)
• Faithful Explanations of Black-box NLP Models Using LLM-generated Counterfactuals [67.64770842323966]
  Causal explanations of predictions of NLP systems are essential to ensure safety and establish trust. Existing methods often fall short of explaining model predictions effectively or efficiently. We propose two approaches for counterfactual (CF) approximation.
  arXiv  Detail & Related papers  (2023-10-01T07:31:04Z)
• Weighted Ensemble Self-Supervised Learning [67.24482854208783]
  Ensembling has proven to be a powerful technique for boosting model performance. We develop a framework that permits data-dependent weighted cross-entropy losses. Our method outperforms both in multiple evaluation metrics on ImageNet-1K.
  arXiv  Detail & Related papers  (2022-11-18T02:00:17Z)
• BigBraveBN: algorithm of structural learning for bayesian networks with a large number of nodes [0.0]
  The article presents a BigBraveBN algorithm for learning large Bayesian Networks with a high number of nodes (over 100) The algorithm utilizes the Brave coefficient that measures the mutual occurrence of instances in several groups. In the experimental part of the article, we compare the performance of BigBraveBN to other existing solutions on multiple data sets both discrete and continuous.
  arXiv  Detail & Related papers  (2022-08-22T13:43:57Z)
• Reconstructing Bayesian Networks on a Quantum Annealer [0.0]
  O'Gorman et al. have proposed an algorithm to encode this task, but they have not provided an experimental evaluation of it. We present (i) an implementation in Python of O'Gorman's algorithm, (ii) a divide et impera approach that allows addressing BNSL problems of larger sizes. Results have shown the effectiveness of O'Gorman's formulation for BNSL instances of small sizes, and the superiority of the divide et impera approach on the direct execution of O'Gorman's algorithm.
  arXiv  Detail & Related papers  (2022-04-07T15:53:05Z)
• Any Part of Bayesian Network Structure Learning [17.46459748913491]
  We study an interesting and challenging problem, learning any part of a Bayesian network (BN) structure. We first present a new concept of Expand-Backtracking to explain why local BN structure learning methods have the false edge orientation problem. We then propose APSL, an efficient and accurate Any Part of BN Structure Learning algorithm.
  arXiv  Detail & Related papers  (2021-03-23T10:03:31Z)
• Sandwich Batch Normalization [96.2529041037824]
  We present Sandwich Batch Normalization (SaBN), an easy improvement of Batch Normalization (BN) with only a few lines of code changes. Our SaBN factorizes the BN affine layer into one shared sandwich affine layer, cascaded by several parallel independent affine layers. We demonstrate the prevailing effectiveness of SaBN as a drop-in replacement in four tasks.
  arXiv  Detail & Related papers  (2021-02-22T22:09:43Z)
• On Resource-Efficient Bayesian Network Classifiers and Deep Neural Networks [14.540226579203207]
  We present two methods to reduce the complexity of Bayesian network (BN) classifiers. First, we introduce quantization-aware training using the straight-through gradient estimator to quantize the parameters of BNs to few bits. Second, we extend a recently proposed differentiable tree-augmented naive Bayes (TAN) structure learning approach by also considering the model size.
  arXiv  Detail & Related papers  (2020-10-22T14:47:55Z)
• Turbocharging Treewidth-Bounded Bayesian Network Structure Learning [26.575053800551633]
  We present a new approach for learning the structure of a treewidth-bounded Network (BN) The key to our approach is applying an exact method (based on MaxSAT) locally to improve the score of aally computed BN. Our experiments show that our method improves the score of BNs provided by state-of-the-art methods, often significantly.
  arXiv  Detail & Related papers  (2020-06-24T16:13:10Z)
• A Generic First-Order Algorithmic Framework for Bi-Level Programming Beyond Lower-Level Singleton [49.23948907229656]
  Bi-level Descent Aggregation is a flexible and modularized algorithmic framework for generic bi-level optimization. We derive a new methodology to prove the convergence of BDA without the LLS condition. Our investigations also demonstrate that BDA is indeed compatible to a verify of particular first-order computation modules.
  arXiv  Detail & Related papers  (2020-06-07T05:18:50Z)
• Towards Stabilizing Batch Statistics in Backward Propagation of Batch Normalization [126.6252371899064]
  Moving Average Batch Normalization (MABN) is a novel normalization method. We show that MABN can completely restore the performance of vanilla BN in small batch cases. Our experiments demonstrate the effectiveness of MABN in multiple computer vision tasks including ImageNet and COCO.
  arXiv  Detail & Related papers  (2020-01-19T14:41:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.