Causal Entropy and Information Gain for Measuring Causal Control

• URL: http://arxiv.org/abs/2309.07703v2
• Date: Fri, 26 Jan 2024 09:55:38 GMT
• Title: Causal Entropy and Information Gain for Measuring Causal Control
• Authors: Francisco Nunes Ferreira Quialheiro Simoes, Mehdi Dastani, Thijs van Ommen
• Abstract summary: We introduce causal versions of entropy and mutual information, termed causal entropy and causal information gain. These quantities capture changes in the entropy of a variable resulting from interventions on other variables. Fundamental results connecting these quantities to the existence of causal effects are derived.
• Score: 0.22252684361733285
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Artificial intelligence models and methods commonly lack causal interpretability. Despite the advancements in interpretable machine learning (IML) methods, they frequently assign importance to features which lack causal influence on the outcome variable. Selecting causally relevant features among those identified as relevant by these methods, or even before model training, would offer a solution. Feature selection methods utilizing information theoretical quantities have been successful in identifying statistically relevant features. However, the information theoretical quantities they are based on do not incorporate causality, rendering them unsuitable for such scenarios. To address this challenge, this article proposes information theoretical quantities that incorporate the causal structure of the system, which can be used to evaluate causal importance of features for some given outcome variable. Specifically, we introduce causal versions of entropy and mutual information, termed causal entropy and causal information gain, which are designed to assess how much control a feature provides over the outcome variable. These newly defined quantities capture changes in the entropy of a variable resulting from interventions on other variables. Fundamental results connecting these quantities to the existence of causal effects are derived. The use of causal information gain in feature selection is demonstrated, highlighting its superiority over standard mutual information in revealing which features provide control over a chosen outcome variable. Our investigation paves the way for the development of methods with improved interpretability in domains involving causation.

Related papers

• Optimal Causal Representations and the Causal Information Bottleneck [0.19799527196428243]
  The Information Bottleneck (IB) method is a widely used approach in representation learning. Traditional methods like IB are purely statistical and ignore underlying causal structures, making them ill-suited for causal tasks. We propose the Causal Information Bottleneck (CIB), a causal extension of the IB, which compresses a set of chosen variables while maintaining causal control over a target variable.
  arXiv  Detail & Related papers  (2024-10-01T09:21:29Z)
• Unifying Causal Representation Learning with the Invariance Principle [21.375611599649716]
  Causal representation learning aims at recovering latent causal variables from high-dimensional observations. Our main contribution is to show that many existing causal representation learning approaches methodologically align the representation to known data symmetries.
  arXiv  Detail & Related papers  (2024-09-04T14:51:36Z)
• Unsupervised Pairwise Causal Discovery on Heterogeneous Data using Mutual Information Measures [49.1574468325115]
  Causal Discovery is a technique that tackles the challenge by analyzing the statistical properties of the constituent variables. We question the current (possibly misleading) baseline results on the basis that they were obtained through supervised learning. In consequence, we approach this problem in an unsupervised way, using robust Mutual Information measures.
  arXiv  Detail & Related papers  (2024-08-01T09:11:08Z)
• Fundamental Properties of Causal Entropy and Information Gain [0.22252684361733285]
  Recent developments enable the quantification of causal control given a structural causal model (SCM) Measures, named causal entropy and causal information gain, aim to address limitations in existing information theoretical approaches for machine learning tasks where causality plays a crucial role.
  arXiv  Detail & Related papers  (2024-02-02T11:55:57Z)
• Causal Feature Selection via Transfer Entropy [59.999594949050596]
  Causal discovery aims to identify causal relationships between features with observational data. We introduce a new causal feature selection approach that relies on the forward and backward feature selection procedures. We provide theoretical guarantees on the regression and classification errors for both the exact and the finite-sample cases.
  arXiv  Detail & Related papers  (2023-10-17T08:04:45Z)
• A Causal Framework for Decomposing Spurious Variations [68.12191782657437]
  We develop tools for decomposing spurious variations in Markovian and Semi-Markovian models. We prove the first results that allow a non-parametric decomposition of spurious effects. The described approach has several applications, ranging from explainable and fair AI to questions in epidemiology and medicine.
  arXiv  Detail & Related papers  (2023-06-08T09:40:28Z)
• Active Bayesian Causal Inference [72.70593653185078]
  We propose Active Bayesian Causal Inference (ABCI), a fully-Bayesian active learning framework for integrated causal discovery and reasoning. ABCI jointly infers a posterior over causal models and queries of interest. We show that our approach is more data-efficient than several baselines that only focus on learning the full causal graph.
  arXiv  Detail & Related papers  (2022-06-04T22:38:57Z)
• Trying to Outrun Causality with Machine Learning: Limitations of Model Explainability Techniques for Identifying Predictive Variables [7.106986689736828]
  We show that machine learning algorithms are not as flexible as they might seem, and are instead incredibly sensitive to the underling causal structure in the data. We provide some alternative recommendations for researchers wanting to explore the data for important variables.
  arXiv  Detail & Related papers  (2022-02-20T17:48:54Z)
• BayesIMP: Uncertainty Quantification for Causal Data Fusion [52.184885680729224]
  We study the causal data fusion problem, where datasets pertaining to multiple causal graphs are combined to estimate the average treatment effect of a target variable. We introduce a framework which combines ideas from probabilistic integration and kernel mean embeddings to represent interventional distributions in the reproducing kernel Hilbert space.
  arXiv  Detail & Related papers  (2021-06-07T10:14:18Z)
• Latent Causal Invariant Model [128.7508609492542]
  Current supervised learning can learn spurious correlation during the data-fitting process. We propose a Latent Causal Invariance Model (LaCIM) which pursues causal prediction.
  arXiv  Detail & Related papers  (2020-11-04T10:00:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.