$χ$SPN: Characteristic Interventional Sum-Product Networks for Causal Inference in Hybrid Domains

• URL: http://arxiv.org/abs/2408.07545v1
• Date: Wed, 14 Aug 2024 13:31:32 GMT
• Title: $χ$SPN: Characteristic Interventional Sum-Product Networks for Causal Inference in Hybrid Domains
• Authors: Harsh Poonia, Moritz Willig, Zhongjie Yu, Matej Zečević, Kristian Kersting, Devendra Singh Dhami,
• Abstract summary: We propose aCharacteristic Interventional Sum-Product Network ($chi$SPN) that is capable of estimating interventional distributions in presence of random variables. $chi$SPN uses characteristic functions in the leaves of an interventional SPN (iSPN) thereby providing a unified view for discrete and continuous random variables. A neural network is used to estimate the parameters of the learned iSPN using the intervened data.
• Score: 19.439265962277716
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Causal inference in hybrid domains, characterized by a mixture of discrete and continuous variables, presents a formidable challenge. We take a step towards this direction and propose Characteristic Interventional Sum-Product Network ($\chi$SPN) that is capable of estimating interventional distributions in presence of random variables drawn from mixed distributions. $\chi$SPN uses characteristic functions in the leaves of an interventional SPN (iSPN) thereby providing a unified view for discrete and continuous random variables through the Fourier-Stieltjes transform of the probability measures. A neural network is used to estimate the parameters of the learned iSPN using the intervened data. Our experiments on 3 synthetic heterogeneous datasets suggest that $\chi$SPN can effectively capture the interventional distributions for both discrete and continuous variables while being expressive and causally adequate. We also show that $\chi$SPN generalize to multiple interventions while being trained only on a single intervention data.

Related papers

• Function Space Diversity for Uncertainty Prediction via Repulsive Last-Layer Ensembles [11.551956337460982]
  We discuss function space inference via particle optimization and present practical modifications that improve uncertainty estimation. In this work, we demonstrate that the input samples, where particle predictions are enforced to be diverse, are detrimental to the model performance. While diversity on training data itself can lead to underfitting, the use of label-destroying data augmentation, or unlabeled out-of-distribution data can improve prediction diversity and uncertainty estimates.
  arXiv  Detail & Related papers  (2024-12-20T10:24:08Z)
• Probabilistic Verification of ReLU Neural Networks via Characteristic Functions [11.489187712465325]
  We use ideas from probability theory in the frequency domain to provide probabilistic verification guarantees for ReLU neural networks. We interpret a (deep) feedforward neural network as a discrete dynamical system over a finite horizon. We obtain the corresponding cumulative distribution function of the output set, which can be used to check if the network is performing as expected.
  arXiv  Detail & Related papers  (2022-12-03T05:53:57Z)
• Exchangeability-Aware Sum-Product Networks [10.506336354512145]
  Sum-Product Networks (SPNs) are expressive probabilistic models that provide exact, tractable inference. The contribution of this paper is a novel probabilistic model which we call Exchangeability-Aware Sum-Product Networks (XSPNs)
  arXiv  Detail & Related papers  (2021-10-11T11:25:31Z)
• Decentralized Local Stochastic Extra-Gradient for Variational Inequalities [125.62877849447729]
  We consider distributed variational inequalities (VIs) on domains with the problem data that is heterogeneous (non-IID) and distributed across many devices. We make a very general assumption on the computational network that covers the settings of fully decentralized calculations. We theoretically analyze its convergence rate in the strongly-monotone, monotone, and non-monotone settings.
  arXiv  Detail & Related papers  (2021-06-15T17:45:51Z)
• Mixture Density Network Estimation of Continuous Variable Maximum Likelihood Using Discrete Training Samples [0.0]
  Mixture Density Networks (MDNs) can be used to generate probability density functions of model parameters $boldsymboltheta$ given a set of observables. We demonstrate the usage of MDNs for parameter estimation, discuss the origins of the biases, and propose a corrective method for each issue.
  arXiv  Detail & Related papers  (2021-03-24T18:02:55Z)
• Sampling-free Variational Inference for Neural Networks with Multiplicative Activation Noise [51.080620762639434]
  We propose a more efficient parameterization of the posterior approximation for sampling-free variational inference. Our approach yields competitive results for standard regression problems and scales well to large-scale image classification tasks.
  arXiv  Detail & Related papers  (2021-03-15T16:16:18Z)
• Interventional Sum-Product Networks: Causal Inference with Tractable Probabilistic Models [26.497268758016595]
  We consider the problem of learning interventional distributions using sum-product networks (SPNs) We provide an arbitrarily intervened causal graph as input, effectively subsuming Pearl's do-operator. The resulting interventional SPNs are motivated and illustrated by a structural causal model themed around personal health.
  arXiv  Detail & Related papers  (2021-02-20T20:48:47Z)
• Probabilistic Circuits for Variational Inference in Discrete Graphical Models [101.28528515775842]
  Inference in discrete graphical models with variational methods is difficult. Many sampling-based methods have been proposed for estimating Evidence Lower Bound (ELBO) We propose a new approach that leverages the tractability of probabilistic circuit models, such as Sum Product Networks (SPN) We show that selective-SPNs are suitable as an expressive variational distribution, and prove that when the log-density of the target model is aweighted the corresponding ELBO can be computed analytically.
  arXiv  Detail & Related papers  (2020-10-22T05:04:38Z)
• Probabilistic Numeric Convolutional Neural Networks [80.42120128330411]
  Continuous input signals like images and time series that are irregularly sampled or have missing values are challenging for existing deep learning methods. We propose Probabilistic Convolutional Neural Networks which represent features as Gaussian processes (GPs) We then define a convolutional layer as the evolution of a PDE defined on this GP, followed by a nonlinearity. In experiments we show that our approach yields a $3times$ reduction of error from the previous state of the art on the SuperPixel-MNIST dataset and competitive performance on the medical time2012 dataset PhysioNet.
  arXiv  Detail & Related papers  (2020-10-21T10:08:21Z)
• GANs with Variational Entropy Regularizers: Applications in Mitigating the Mode-Collapse Issue [95.23775347605923]
  Building on the success of deep learning, Generative Adversarial Networks (GANs) provide a modern approach to learn a probability distribution from observed samples. GANs often suffer from the mode collapse issue where the generator fails to capture all existing modes of the input distribution. We take an information-theoretic approach and maximize a variational lower bound on the entropy of the generated samples to increase their diversity.
  arXiv  Detail & Related papers  (2020-09-24T19:34:37Z)
• NP-PROV: Neural Processes with Position-Relevant-Only Variances [113.20013269514327]
  We present a new member named Neural Processes with Position-Relevant-Only Variances (NP-PROV) NP-PROV hypothesizes that a target point close to a context point has small uncertainty, regardless of the function value at that position. Our evaluation on synthetic and real-world datasets reveals that NP-PROV can achieve state-of-the-art likelihood while retaining a bounded variance.
  arXiv  Detail & Related papers  (2020-06-15T06:11:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.