Bivariate Causal Discovery Using Rate-Distortion MDL: An Information Dimension Approach

• URL: http://arxiv.org/abs/2604.05829v1
• Date: Tue, 07 Apr 2026 13:01:49 GMT
• Title: Bivariate Causal Discovery Using Rate-Distortion MDL: An Information Dimension Approach
• Abstract summary: We argue that current state-of-the-art MDL-based methods do not correctly address the problem of estimating the description length of the cause variable.<n>Based on rate-distortion theory, we propose a new way to measure the description length of the cause, corresponding to the minimum rate required to achieve a distortion level representative of the underlying distribution.
• Score: 2.1986675261317163
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Approaches to bivariate causal discovery based on the minimum description length (MDL) principle approximate the (uncomputable) Kolmogorov complexity of the models in each causal direction, selecting the one with the lower total complexity. The premise is that nature's mechanisms are simpler in their true causal order. Inherently, the description length (complexity) in each direction includes the description of the cause variable and that of the causal mechanism. In this work, we argue that current state-of-the-art MDL-based methods do not correctly address the problem of estimating the description length of the cause variable, effectively leaving the decision to the description length of the causal mechanism. Based on rate-distortion theory, we propose a new way to measure the description length of the cause, corresponding to the minimum rate required to achieve a distortion level representative of the underlying distribution. This distortion level is deduced using rules from histogram-based density estimation, while the rate is computed using the related concept of information dimension, based on an asymptotic approximation. Combining it with a traditional approach for the causal mechanism, we introduce a new bivariate causal discovery method, termed rate-distortion MDL (RDMDL). We show experimentally that RDMDL achieves competitive performance on the Tübingen dataset. All the code and experiments are publicly available at github.com/tiagobrogueira/Causal-Discovery-In-Exchangeable-Data.

Related papers

• Generation Order and Parallel Decoding in Masked Diffusion Models: An Information-Theoretic Perspective [16.942478643768144]
  Masked Diffusion Models (MDMs) significantly accelerate inference by trading off sequential determinism.<n>We provide a unified information-theoretic framework to decouple and analyze two fundamental sources of failure: order sensitivity and parallelization bias.
  arXiv  Detail & Related papers  (2026-01-30T20:15:18Z)
• Learning Causal States Under Partial Observability and Perturbation [29.533770208192845]
  Existing methods fail to mitigate perturbations while addressing partial observability.<n>We propose textitCausal State Representation under Asynchronous Diffusion Model (CaDiff)<n>CaDiff is the first framework that approximates causal states using diffusion models with both theoretical rigor and practicality.
  arXiv  Detail & Related papers  (2025-11-29T06:56:03Z)
• Towards Minimal Causal Representations for Human Multimodal Language Understanding [20.44307628909198]
  We introduce a Causal Multimodal Information Bottleneck (CaMIB) model that leverages causal principles rather than traditional likelihood.<n>To ensure global consistency of causal features, we incorporate an instrumental variable constraint.<n>Experiments on multimodal sentiment analysis, humor detection, and sarcasm detection, along with OOD test sets, demonstrate the effectiveness of CaMIB.
  arXiv  Detail & Related papers  (2025-09-26T03:04:23Z)
• Identifying Causal Direction via Dense Functional Classes [0.0]
  We show that LCUBE achieves superior precision in terms of AUDRC on the real-world Tuebingen cause-effect pairs dataset.<n>It also shows superior average precision across common 10 benchmark datasets and achieves above average precision on 13 datasets.
  arXiv  Detail & Related papers  (2025-08-30T15:42:31Z)
• Model-free Methods for Event History Analysis and Efficient Adjustment (PhD Thesis) [55.2480439325792]
  This thesis is a series of independent contributions to statistics unified by a model-free perspective.<n>The first chapter elaborates on how a model-free perspective can be used to formulate flexible methods that leverage prediction techniques from machine learning.<n>The second chapter studies the concept of local independence, which describes whether the evolution of one process is directly influenced by another.
  arXiv  Detail & Related papers  (2025-02-11T19:24:09Z)
• Estimation of Bivariate Structural Causal Models by Variational Gaussian Process Regression Under Likelihoods Parametrised by Normalising Flows [74.85071867225533]
  Causal mechanisms can be described by structural causal models. One major drawback of state-of-the-art artificial intelligence is its lack of explainability.
  arXiv  Detail & Related papers  (2021-09-06T14:52:58Z)
• Variational Causal Networks: Approximate Bayesian Inference over Causal Structures [132.74509389517203]
  We introduce a parametric variational family modelled by an autoregressive distribution over the space of discrete DAGs. In experiments, we demonstrate that the proposed variational posterior is able to provide a good approximation of the true posterior.
  arXiv  Detail & Related papers  (2021-06-14T17:52:49Z)
• Counterfactual Maximum Likelihood Estimation for Training Deep Networks [83.44219640437657]
  Deep learning models are prone to learning spurious correlations that should not be learned as predictive clues. We propose a causality-based training framework to reduce the spurious correlations caused by observable confounders. We conduct experiments on two real-world tasks: Natural Language Inference (NLI) and Image Captioning.
  arXiv  Detail & Related papers  (2021-06-07T17:47:16Z)
• Efficient Causal Inference from Combined Observational and Interventional Data through Causal Reductions [68.6505592770171]
  Unobserved confounding is one of the main challenges when estimating causal effects. We propose a novel causal reduction method that replaces an arbitrary number of possibly high-dimensional latent confounders. We propose a learning algorithm to estimate the parameterized reduced model jointly from observational and interventional data.
  arXiv  Detail & Related papers  (2021-03-08T14:29:07Z)
• Scalable Vector Gaussian Information Bottleneck [19.21005180893519]
  We study a variation of the problem, called scalable information bottleneck, in which the encoder outputs multiple descriptions of the observation. We derive a variational inference type algorithm for general sources with unknown distribution; and show means of parametrizing it using neural networks.
  arXiv  Detail & Related papers  (2021-02-15T12:51:26Z)
• A Critical View of the Structural Causal Model [89.43277111586258]
  We show that one can identify the cause and the effect without considering their interaction at all. We propose a new adversarial training method that mimics the disentangled structure of the causal model. Our multidimensional method outperforms the literature methods on both synthetic and real world datasets.
  arXiv  Detail & Related papers  (2020-02-23T22:52:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.