Quantifying Consistency and Information Loss for Causal Abstraction Learning

• URL: http://arxiv.org/abs/2305.04357v1
• Date: Sun, 7 May 2023 19:10:28 GMT
• Title: Quantifying Consistency and Information Loss for Causal Abstraction Learning
• Authors: Fabio Massimo Zennaro, Paolo Turrini, Theodoros Damoulas
• Abstract summary: We introduce a family of interventional measures that an agent may use to evaluate such a trade-off. We consider four measures suited for different tasks, analyze their properties, and propose algorithms to evaluate and learn causal abstractions.
• Score: 16.17846886492361
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Structural causal models provide a formalism to express causal relations between variables of interest. Models and variables can represent a system at different levels of abstraction, whereby relations may be coarsened and refined according to the need of a modeller. However, switching between different levels of abstraction requires evaluating a trade-off between the consistency and the information loss among different models. In this paper we introduce a family of interventional measures that an agent may use to evaluate such a trade-off. We consider four measures suited for different tasks, analyze their properties, and propose algorithms to evaluate and learn causal abstractions. Finally, we illustrate the flexibility of our setup by empirically showing how different measures and algorithmic choices may lead to different abstractions.

Related papers

• Abstraction Alignment: Comparing Model and Human Conceptual Relationships [26.503178592074757]
  We introduce abstraction alignment, a methodology to measure the agreement between a model's learned abstraction and the expected human abstraction. In evaluation tasks, abstraction alignment provides a deeper understanding of model behavior and dataset content.
  arXiv  Detail & Related papers  (2024-07-17T13:27:26Z)
• Building Minimal and Reusable Causal State Abstractions for Reinforcement Learning [63.58935783293342]
  Causal Bisimulation Modeling (CBM) is a method that learns the causal relationships in the dynamics and reward functions for each task to derive a minimal, task-specific abstraction. CBM's learned implicit dynamics models identify the underlying causal relationships and state abstractions more accurately than explicit ones.
  arXiv  Detail & Related papers  (2024-01-23T05:43:15Z)
• Flow Factorized Representation Learning [109.51947536586677]
  We introduce a generative model which specifies a distinct set of latent probability paths that define different input transformations. We show that our model achieves higher likelihoods on standard representation learning benchmarks while simultaneously being closer to approximately equivariant models.
  arXiv  Detail & Related papers  (2023-09-22T20:15:37Z)
• Inducing Causal Structure for Abstractive Text Summarization [76.1000380429553]
  We introduce a Structural Causal Model (SCM) to induce the underlying causal structure of the summarization data. We propose a Causality Inspired Sequence-to-Sequence model (CI-Seq2Seq) to learn the causal representations that can mimic the causal factors. Experimental results on two widely used text summarization datasets demonstrate the advantages of our approach.
  arXiv  Detail & Related papers  (2023-08-24T16:06:36Z)
• Towards Computing an Optimal Abstraction for Structural Causal Models [16.17846886492361]
  We focus on the problem of learning abstractions. We suggest a concrete measure of information loss, and we illustrate its contribution to learning new abstractions.
  arXiv  Detail & Related papers  (2022-08-01T14:35:57Z)
• Abstraction between Structural Causal Models: A Review of Definitions and Properties [0.0]
  Structural causal models (SCMs) are a widespread formalism to deal with causal systems. This paper focuses on the formal properties of a map between SCMs, and highlighting the different layers (structural, distributional) at which these properties may be enforced.
  arXiv  Detail & Related papers  (2022-07-18T13:47:20Z)
• Discriminative Multimodal Learning via Conditional Priors in Generative Models [21.166519800652047]
  This research studies the realistic scenario in which all modalities and class labels are available for model training. We show, in this scenario, that the variational lower bound limits mutual information between joint representations and missing modalities.
  arXiv  Detail & Related papers  (2021-10-09T17:22:24Z)
• Model-agnostic multi-objective approach for the evolutionary discovery of mathematical models [55.41644538483948]
  In modern data science, it is more interesting to understand the properties of the model, which parts could be replaced to obtain better results. We use multi-objective evolutionary optimization for composite data-driven model learning to obtain the algorithm's desired properties.
  arXiv  Detail & Related papers  (2021-07-07T11:17:09Z)
• MINIMALIST: Mutual INformatIon Maximization for Amortized Likelihood Inference from Sampled Trajectories [61.3299263929289]
  Simulation-based inference enables learning the parameters of a model even when its likelihood cannot be computed in practice. One class of methods uses data simulated with different parameters to infer an amortized estimator for the likelihood-to-evidence ratio. We show that this approach can be formulated in terms of mutual information between model parameters and simulated data.
  arXiv  Detail & Related papers  (2021-06-03T12:59:16Z)
• Improving the Reconstruction of Disentangled Representation Learners via Multi-Stage Modeling [54.94763543386523]
  Current autoencoder-based disentangled representation learning methods achieve disentanglement by penalizing the ( aggregate) posterior to encourage statistical independence of the latent factors. We present a novel multi-stage modeling approach where the disentangled factors are first learned using a penalty-based disentangled representation learning method. Then, the low-quality reconstruction is improved with another deep generative model that is trained to model the missing correlated latent variables.
  arXiv  Detail & Related papers  (2020-10-25T18:51:15Z)
• Learning Structured Latent Factors from Dependent Data:A Generative Model Framework from Information-Theoretic Perspective [18.88255368184596]
  We present a novel framework for learning generative models with various underlying structures in the latent space. Our model provides a principled approach to learn a set of semantically meaningful latent factors that reflect various types of desired structures.
  arXiv  Detail & Related papers  (2020-07-21T06:59:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.