Hard and Soft EM in Bayesian Network Learning from Incomplete Data

• URL: http://arxiv.org/abs/2012.05269v1
• Date: Wed, 9 Dec 2020 19:13:32 GMT
• Title: Hard and Soft EM in Bayesian Network Learning from Incomplete Data
• Authors: Andrea Ruggieri, Francesco Stranieri, Fabio Stella and Marco Scutari
• Abstract summary: We investigate the impact of using imputation instead of belief propagation on the quality of the resulting BNs. We find that it is possible to recommend one approach over the other in several scenarios based on the characteristics of the data.
• Score: 1.5484595752241122
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Incomplete data are a common feature in many domains, from clinical trials to industrial applications. Bayesian networks (BNs) are often used in these domains because of their graphical and causal interpretations. BN parameter learning from incomplete data is usually implemented with the Expectation-Maximisation algorithm (EM), which computes the relevant sufficient statistics ("soft EM") using belief propagation. Similarly, the Structural Expectation-Maximisation algorithm (Structural EM) learns the network structure of the BN from those sufficient statistics using algorithms designed for complete data. However, practical implementations of parameter and structure learning often impute missing data ("hard EM") to compute sufficient statistics instead of using belief propagation, for both ease of implementation and computational speed. In this paper, we investigate the question: what is the impact of using imputation instead of belief propagation on the quality of the resulting BNs? From a simulation study using synthetic data and reference BNs, we find that it is possible to recommend one approach over the other in several scenarios based on the characteristics of the data. We then use this information to build a simple decision tree to guide practitioners in choosing the EM algorithm best suited to their problem.

Related papers

• Rethinking Deep Learning: Propagating Information in Neural Networks without Backpropagation and Statistical Optimization [0.0]
  This study discusses the information propagation capabilities and potential practical applications of NNs as neural system mimicking structures. In this study, the NNs architecture comprises fully connected layers using step functions as activation functions, with 0-15 hidden layers, and no weight updates. The accuracy is calculated by comparing the average output vectors of the training data for each label with the output vectors of the test data, based on vector similarity.
  arXiv  Detail & Related papers  (2024-08-18T09:22:24Z)
• Minimally Supervised Learning using Topological Projections in Self-Organizing Maps [55.31182147885694]
  We introduce a semi-supervised learning approach based on topological projections in self-organizing maps (SOMs) Our proposed method first trains SOMs on unlabeled data and then a minimal number of available labeled data points are assigned to key best matching units (BMU) Our results indicate that the proposed minimally supervised model significantly outperforms traditional regression techniques.
  arXiv  Detail & Related papers  (2024-01-12T22:51:48Z)
• Fast & Efficient Learning of Bayesian Networks from Data: Knowledge Discovery and Causality [0.0]
  Two novel algorithms, FSBN and SSBN, employ local search strategy and conditional independence tests to learn the causal network structure from data. FSBN achieves up to 52% cost reduction, while SSBN surpasses it with a remarkable 72% reduction for a 200-node network.
  arXiv  Detail & Related papers  (2023-10-13T16:20:20Z)
• Large-Scale OD Matrix Estimation with A Deep Learning Method [70.78575952309023]
  The proposed method integrates deep learning and numerical optimization algorithms to infer matrix structure and guide numerical optimization. We conducted tests to demonstrate the good generalization performance of our method on a large-scale synthetic dataset.
  arXiv  Detail & Related papers  (2023-10-09T14:30:06Z)
• Learning to Bound Counterfactual Inference in Structural Causal Models from Observational and Randomised Data [64.96984404868411]
  We derive a likelihood characterisation for the overall data that leads us to extend a previous EM-based algorithm. The new algorithm learns to approximate the (unidentifiability) region of model parameters from such mixed data sources. It delivers interval approximations to counterfactual results, which collapse to points in the identifiable case.
  arXiv  Detail & Related papers  (2022-12-06T12:42:11Z)
• BigBraveBN: algorithm of structural learning for bayesian networks with a large number of nodes [0.0]
  The article presents a BigBraveBN algorithm for learning large Bayesian Networks with a high number of nodes (over 100) The algorithm utilizes the Brave coefficient that measures the mutual occurrence of instances in several groups. In the experimental part of the article, we compare the performance of BigBraveBN to other existing solutions on multiple data sets both discrete and continuous.
  arXiv  Detail & Related papers  (2022-08-22T13:43:57Z)
• A Differentiable Approach to Combinatorial Optimization using Dataless Neural Networks [20.170140039052455]
  We propose a radically different approach in that no data is required for training the neural networks that produce the solution. In particular, we reduce the optimization problem to a neural network and employ a dataless training scheme to refine the parameters of the network such that those parameters yield the structure of interest.
  arXiv  Detail & Related papers  (2022-03-15T19:21:31Z)
• Solving Mixed Integer Programs Using Neural Networks [57.683491412480635]
  This paper applies learning to the two key sub-tasks of a MIP solver, generating a high-quality joint variable assignment, and bounding the gap in objective value between that assignment and an optimal one. Our approach constructs two corresponding neural network-based components, Neural Diving and Neural Branching, to use in a base MIP solver such as SCIP. We evaluate our approach on six diverse real-world datasets, including two Google production datasets and MIPLIB, by training separate neural networks on each.
  arXiv  Detail & Related papers  (2020-12-23T09:33:11Z)
• A Constraint-Based Algorithm for the Structural Learning of Continuous-Time Bayesian Networks [70.88503833248159]
  We propose the first constraint-based algorithm for learning the structure of continuous-time Bayesian networks. We discuss the different statistical tests and the underlying hypotheses used by our proposal to establish conditional independence.
  arXiv  Detail & Related papers  (2020-07-07T07:34:09Z)
• FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Data [59.50904660420082]
  Federated Learning (FL) has become a popular paradigm for learning from distributed data. To effectively utilize data at different devices without moving them to the cloud, algorithms such as the Federated Averaging (FedAvg) have adopted a "computation then aggregation" (CTA) model.
  arXiv  Detail & Related papers  (2020-05-22T23:07:42Z)
• Large-scale empirical validation of Bayesian Network structure learning algorithms with noisy data [9.04391541965756]
  This paper investigates the performance of 15 structure learning algorithms. Each algorithm is tested over multiple case studies, sample sizes, types of noise, and assessed with multiple evaluation criteria. Results suggest traditional synthetic performance may overestimate real-world performance by anywhere between 10% and more than 50%.
  arXiv  Detail & Related papers  (2020-05-18T18:40:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.