Causal Entropy and Information Gain for Measuring Causal Control
- URL: http://arxiv.org/abs/2309.07703v2
- Date: Fri, 26 Jan 2024 09:55:38 GMT
- Title: Causal Entropy and Information Gain for Measuring Causal Control
- Authors: Francisco Nunes Ferreira Quialheiro Simoes, Mehdi Dastani, Thijs van
Ommen
- Abstract summary: We introduce causal versions of entropy and mutual information, termed causal entropy and causal information gain.
These quantities capture changes in the entropy of a variable resulting from interventions on other variables.
Fundamental results connecting these quantities to the existence of causal effects are derived.
- Score: 0.22252684361733285
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Artificial intelligence models and methods commonly lack causal
interpretability. Despite the advancements in interpretable machine learning
(IML) methods, they frequently assign importance to features which lack causal
influence on the outcome variable. Selecting causally relevant features among
those identified as relevant by these methods, or even before model training,
would offer a solution. Feature selection methods utilizing information
theoretical quantities have been successful in identifying statistically
relevant features. However, the information theoretical quantities they are
based on do not incorporate causality, rendering them unsuitable for such
scenarios. To address this challenge, this article proposes information
theoretical quantities that incorporate the causal structure of the system,
which can be used to evaluate causal importance of features for some given
outcome variable. Specifically, we introduce causal versions of entropy and
mutual information, termed causal entropy and causal information gain, which
are designed to assess how much control a feature provides over the outcome
variable. These newly defined quantities capture changes in the entropy of a
variable resulting from interventions on other variables. Fundamental results
connecting these quantities to the existence of causal effects are derived. The
use of causal information gain in feature selection is demonstrated,
highlighting its superiority over standard mutual information in revealing
which features provide control over a chosen outcome variable. Our
investigation paves the way for the development of methods with improved
interpretability in domains involving causation.
Related papers
- Fundamental Properties of Causal Entropy and Information Gain [0.22252684361733285]
Recent developments enable the quantification of causal control given a structural causal model (SCM)
Measures, named causal entropy and causal information gain, aim to address limitations in existing information theoretical approaches for machine learning tasks where causality plays a crucial role.
arXiv Detail & Related papers (2024-02-02T11:55:57Z) - Causal Feature Selection via Transfer Entropy [59.999594949050596]
Causal discovery aims to identify causal relationships between features with observational data.
We introduce a new causal feature selection approach that relies on the forward and backward feature selection procedures.
We provide theoretical guarantees on the regression and classification errors for both the exact and the finite-sample cases.
arXiv Detail & Related papers (2023-10-17T08:04:45Z) - A Causal Framework for Decomposing Spurious Variations [68.12191782657437]
We develop tools for decomposing spurious variations in Markovian and Semi-Markovian models.
We prove the first results that allow a non-parametric decomposition of spurious effects.
The described approach has several applications, ranging from explainable and fair AI to questions in epidemiology and medicine.
arXiv Detail & Related papers (2023-06-08T09:40:28Z) - Nonparametric Identifiability of Causal Representations from Unknown
Interventions [63.1354734978244]
We study causal representation learning, the task of inferring latent causal variables and their causal relations from mixtures of the variables.
Our goal is to identify both the ground truth latents and their causal graph up to a set of ambiguities which we show to be irresolvable from interventional data.
arXiv Detail & Related papers (2023-06-01T10:51:58Z) - Active Bayesian Causal Inference [72.70593653185078]
We propose Active Bayesian Causal Inference (ABCI), a fully-Bayesian active learning framework for integrated causal discovery and reasoning.
ABCI jointly infers a posterior over causal models and queries of interest.
We show that our approach is more data-efficient than several baselines that only focus on learning the full causal graph.
arXiv Detail & Related papers (2022-06-04T22:38:57Z) - Trying to Outrun Causality with Machine Learning: Limitations of Model
Explainability Techniques for Identifying Predictive Variables [7.106986689736828]
We show that machine learning algorithms are not as flexible as they might seem, and are instead incredibly sensitive to the underling causal structure in the data.
We provide some alternative recommendations for researchers wanting to explore the data for important variables.
arXiv Detail & Related papers (2022-02-20T17:48:54Z) - BayesIMP: Uncertainty Quantification for Causal Data Fusion [52.184885680729224]
We study the causal data fusion problem, where datasets pertaining to multiple causal graphs are combined to estimate the average treatment effect of a target variable.
We introduce a framework which combines ideas from probabilistic integration and kernel mean embeddings to represent interventional distributions in the reproducing kernel Hilbert space.
arXiv Detail & Related papers (2021-06-07T10:14:18Z) - Latent Causal Invariant Model [128.7508609492542]
Current supervised learning can learn spurious correlation during the data-fitting process.
We propose a Latent Causal Invariance Model (LaCIM) which pursues causal prediction.
arXiv Detail & Related papers (2020-11-04T10:00:27Z) - Estimating Causal Effects with the Neural Autoregressive Density
Estimator [6.59529078336196]
We use neural autoregressive density estimators to estimate causal effects within the Pearl's do-calculus framework.
We show that the approach can retrieve causal effects from non-linear systems without explicitly modeling the interactions between the variables.
arXiv Detail & Related papers (2020-08-17T13:12:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.