Fundamental Properties of Causal Entropy and Information Gain

• URL: http://arxiv.org/abs/2402.01341v2
• Date: Mon, 19 Feb 2024 14:20:56 GMT
• Title: Fundamental Properties of Causal Entropy and Information Gain
• Authors: Francisco N. F. Q. Simoes, Mehdi Dastani, Thijs van Ommen
• Abstract summary: Recent developments enable the quantification of causal control given a structural causal model (SCM) Measures, named causal entropy and causal information gain, aim to address limitations in existing information theoretical approaches for machine learning tasks where causality plays a crucial role.
• Score: 0.22252684361733285
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent developments enable the quantification of causal control given a structural causal model (SCM). This has been accomplished by introducing quantities which encode changes in the entropy of one variable when intervening on another. These measures, named causal entropy and causal information gain, aim to address limitations in existing information theoretical approaches for machine learning tasks where causality plays a crucial role. They have not yet been properly mathematically studied. Our research contributes to the formal understanding of the notions of causal entropy and causal information gain by establishing and analyzing fundamental properties of these concepts, including bounds and chain rules. Furthermore, we elucidate the relationship between causal entropy and stochastic interventions. We also propose definitions for causal conditional entropy and causal conditional information gain. Overall, this exploration paves the way for enhancing causal machine learning tasks through the study of recently-proposed information theoretic quantities grounded in considerations about causality.

Related papers

• Emergence and Causality in Complex Systems: A Survey on Causal Emergence and Related Quantitative Studies [12.78006421209864]
  Causal emergence theory employs measures of causality to quantify emergence. Two key problems are addressed: quantifying causal emergence and identifying it in data. We highlighted that the architectures used for identifying causal emergence are shared by causal representation learning, causal model abstraction, and world model-based reinforcement learning.
  arXiv  Detail & Related papers  (2023-12-28T04:20:46Z)
• Identifiable Latent Polynomial Causal Models Through the Lens of Change [82.14087963690561]
  Causal representation learning aims to unveil latent high-level causal representations from observed low-level data. One of its primary tasks is to provide reliable assurance of identifying these latent causal models, known as identifiability.
  arXiv  Detail & Related papers  (2023-10-24T07:46:10Z)
• Causal Entropy and Information Gain for Measuring Causal Control [0.22252684361733285]
  We introduce causal versions of entropy and mutual information, termed causal entropy and causal information gain. These quantities capture changes in the entropy of a variable resulting from interventions on other variables. Fundamental results connecting these quantities to the existence of causal effects are derived.
  arXiv  Detail & Related papers  (2023-09-14T13:25:42Z)
• A Causal Framework for Decomposing Spurious Variations [68.12191782657437]
  We develop tools for decomposing spurious variations in Markovian and Semi-Markovian models. We prove the first results that allow a non-parametric decomposition of spurious effects. The described approach has several applications, ranging from explainable and fair AI to questions in epidemiology and medicine.
  arXiv  Detail & Related papers  (2023-06-08T09:40:28Z)
• A Survey on Causal Discovery: Theory and Practice [2.741266294612776]
  Causal inference is designed to quantify the underlying relationships that connect a cause to its effect. In this paper, we explore recent advancements in a unified manner, provide a consistent overview of existing algorithms, report useful tools and data, present real-world applications.
  arXiv  Detail & Related papers  (2023-05-17T08:18:56Z)
• Quantify the Causes of Causal Emergence: Critical Conditions of Uncertainty and Asymmetry in Causal Structure [0.5372002358734439]
  Investigation of causal relationships based on statistical and informational theories have posed an interesting and valuable challenge to large-scale models. This paper introduces a framework for assessing numerical conditions of Causal Emergence as theoretical constraints of its occurrence.
  arXiv  Detail & Related papers  (2022-12-03T06:35:54Z)
• Variational Causal Networks: Approximate Bayesian Inference over Causal Structures [132.74509389517203]
  We introduce a parametric variational family modelled by an autoregressive distribution over the space of discrete DAGs. In experiments, we demonstrate that the proposed variational posterior is able to provide a good approximation of the true posterior.
  arXiv  Detail & Related papers  (2021-06-14T17:52:49Z)
• BayesIMP: Uncertainty Quantification for Causal Data Fusion [52.184885680729224]
  We study the causal data fusion problem, where datasets pertaining to multiple causal graphs are combined to estimate the average treatment effect of a target variable. We introduce a framework which combines ideas from probabilistic integration and kernel mean embeddings to represent interventional distributions in the reproducing kernel Hilbert space.
  arXiv  Detail & Related papers  (2021-06-07T10:14:18Z)
• Everything Has a Cause: Leveraging Causal Inference in Legal Text Analysis [62.44432226563088]
  Causal inference is the process of capturing cause-effect relationship among variables. We propose a novel Graph-based Causal Inference framework, which builds causal graphs from fact descriptions without much human involvement. We observe that the causal knowledge contained in GCI can be effectively injected into powerful neural networks for better performance and interpretability.
  arXiv  Detail & Related papers  (2021-04-19T16:13:10Z)
• Towards Causal Representation Learning [96.110881654479]
  The two fields of machine learning and graphical causality arose and developed separately. There is now cross-pollination and increasing interest in both fields to benefit from the advances of the other.
  arXiv  Detail & Related papers  (2021-02-22T15:26:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.