Causal learning with sufficient statistics: an information bottleneck approach

• URL: http://arxiv.org/abs/2010.05375v1
• Date: Mon, 12 Oct 2020 00:20:01 GMT
• Title: Causal learning with sufficient statistics: an information bottleneck approach
• Authors: Daniel Chicharro, Michel Besserve, Stefano Panzeri
• Abstract summary: Methods extracting causal information from conditional independencies between variables of a system are common. We capitalize on the fact that the laws governing the generative mechanisms of a system often result in substructures embodied in the generative functional equation of a variable. We propose to use the Information Bottleneck method, a technique commonly applied for dimensionality reduction, to find underlying sufficient sets of statistics.
• Score: 3.720546514089338
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The inference of causal relationships using observational data from partially observed multivariate systems with hidden variables is a fundamental question in many scientific domains. Methods extracting causal information from conditional independencies between variables of a system are common tools for this purpose, but are limited in the lack of independencies. To surmount this limitation, we capitalize on the fact that the laws governing the generative mechanisms of a system often result in substructures embodied in the generative functional equation of a variable, which act as sufficient statistics for the influence that other variables have on it. These functional sufficient statistics constitute intermediate hidden variables providing new conditional independencies to be tested. We propose to use the Information Bottleneck method, a technique commonly applied for dimensionality reduction, to find underlying sufficient sets of statistics. Using these statistics we formulate new additional rules of causal orientation that provide causal information not obtainable from standard structure learning algorithms, which exploit only conditional independencies between observable variables. We validate the use of sufficient statistics for structure learning both with simulated systems built to contain specific sufficient statistics and with benchmark data from regulatory rules previously and independently proposed to model biological signal transduction networks.

Related papers

• Targeted Cause Discovery with Data-Driven Learning [66.86881771339145]
  We propose a novel machine learning approach for inferring causal variables of a target variable from observations. We employ a neural network trained to identify causality through supervised learning on simulated data. Empirical results demonstrate the effectiveness of our method in identifying causal relationships within large-scale gene regulatory networks.
  arXiv  Detail & Related papers  (2024-08-29T02:21:11Z)
• Generalization Bounds for Dependent Data using Online-to-Batch Conversion [0.6144680854063935]
  We show that the generalization error of statistical learners in the dependent data setting is equivalent to the generalization error of statistical learners in the i.i.d. setting. Our proof techniques involve defining a new notion of stability of online learning algorithms based on Wasserstein.
  arXiv  Detail & Related papers  (2024-05-22T14:07:25Z)
• DCID: Deep Canonical Information Decomposition [84.59396326810085]
  We consider the problem of identifying the signal shared between two one-dimensional target variables. We propose ICM, an evaluation metric which can be used in the presence of ground-truth labels. We also propose Deep Canonical Information Decomposition (DCID) - a simple, yet effective approach for learning the shared variables.
  arXiv  Detail & Related papers  (2023-06-27T16:59:06Z)
• Conditional expectation with regularization for missing data imputation [19.254291863337347]
  Missing data frequently occurs in datasets across various domains, such as medicine, sports, and finance. We propose a new algorithm named "conditional Distribution-based Imputation of Missing Values with Regularization" (DIMV) DIMV operates by determining the conditional distribution of a feature that has missing entries, using the information from the fully observed features as a basis.
  arXiv  Detail & Related papers  (2023-02-02T06:59:15Z)
• Measuring Statistical Dependencies via Maximum Norm and Characteristic Functions [0.0]
  We propose a statistical dependence measure based on the maximum-norm of the difference between joint and product-marginal characteristic functions. The proposed measure can detect arbitrary statistical dependence between two random vectors of possibly different dimensions. We conduct experiments both with simulated and real data.
  arXiv  Detail & Related papers  (2022-08-16T20:24:31Z)
• Causality-Based Multivariate Time Series Anomaly Detection [63.799474860969156]
  We formulate the anomaly detection problem from a causal perspective and view anomalies as instances that do not follow the regular causal mechanism to generate the multivariate data. We then propose a causality-based anomaly detection approach, which first learns the causal structure from data and then infers whether an instance is an anomaly relative to the local causal mechanism. We evaluate our approach with both simulated and public datasets as well as a case study on real-world AIOps applications.
  arXiv  Detail & Related papers  (2022-06-30T06:00:13Z)
• Equivariance Allows Handling Multiple Nuisance Variables When Analyzing Pooled Neuroimaging Datasets [53.34152466646884]
  In this paper, we show how bringing recent results on equivariant representation learning instantiated on structured spaces together with simple use of classical results on causal inference provides an effective practical solution. We demonstrate how our model allows dealing with more than one nuisance variable under some assumptions and can enable analysis of pooled scientific datasets in scenarios that would otherwise entail removing a large portion of the samples.
  arXiv  Detail & Related papers  (2022-03-29T04:54:06Z)
• Measuring Dependence with Matrix-based Entropy Functional [21.713076360132195]
  Measuring the dependence of data plays a central role in statistics and machine learning. We propose two measures, namely the matrix-based normalized total correlation ($T_alpha*$) and the matrix-based normalized dual total correlation ($D_alpha*$) We show that our measures are differentiable and statistically more powerful than prevalent ones.
  arXiv  Detail & Related papers  (2021-01-25T15:18:16Z)
• Estimating Structural Target Functions using Machine Learning and Influence Functions [103.47897241856603]
  We propose a new framework for statistical machine learning of target functions arising as identifiable functionals from statistical models. This framework is problem- and model-agnostic and can be used to estimate a broad variety of target parameters of interest in applied statistics. We put particular focus on so-called coarsening at random/doubly robust problems with partially unobserved information.
  arXiv  Detail & Related papers  (2020-08-14T16:48:29Z)
• Measuring the Discrepancy between Conditional Distributions: Methods, Properties and Applications [18.293397644865458]
  We propose a simple yet powerful test statistic to quantify the discrepancy between two conditional distributions. The new statistic avoids the explicit estimation of the underlying distributions in highdimensional space. It inherits the merits of the correntropy function to explicitly incorporate high-order statistics in the data.
  arXiv  Detail & Related papers  (2020-05-05T14:03:55Z)
• The empirical duality gap of constrained statistical learning [115.23598260228587]
  We study the study of constrained statistical learning problems, the unconstrained version of which are at the core of virtually all modern information processing. We propose to tackle the constrained statistical problem overcoming its infinite dimensionality, unknown distributions, and constraints by leveraging finite dimensional parameterizations, sample averages, and duality theory. We demonstrate the effectiveness and usefulness of this constrained formulation in a fair learning application.
  arXiv  Detail & Related papers  (2020-02-12T19:12:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.