De-paradox Tree: Breaking Down Simpson's Paradox via A Kernel-Based Partition Algorithm

• URL: http://arxiv.org/abs/2603.02174v1
• Date: Mon, 02 Mar 2026 18:45:24 GMT
• Title: De-paradox Tree: Breaking Down Simpson's Paradox via A Kernel-Based Partition Algorithm
• Authors: Xian Teng, Yu-Ru Lin,
• Abstract summary: Simpson's paradox exemplifies this challenge, where aggregated and subgroup-level associations contradict each other.<n>We introduce De-paradox Tree, an interpretable algorithm designed to uncover hidden subgroup patterns behind paradoxical associations.<n>Our approach addresses the limitations of traditional causal inference and machine learning methods by introducing an interpretable framework.
• Score: 3.566568169425391
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Real-world observational datasets and machine learning have revolutionized data-driven decision-making, yet many models rely on empirical associations that may be misleading due to confounding and subgroup heterogeneity. Simpson's paradox exemplifies this challenge, where aggregated and subgroup-level associations contradict each other, leading to misleading conclusions. Existing methods provide limited support for detecting and interpreting such paradoxical associations, especially for practitioners without deep causal expertise. We introduce De-paradox Tree, an interpretable algorithm designed to uncover hidden subgroup patterns behind paradoxical associations under assumed causal structures involving confounders and effect heterogeneity. It employs novel split criteria and balancing-based procedures to adjust for confounders and homogenize heterogeneous effects through recursive partitioning. Compared to state-of-the-art methods, De-paradox Tree builds simpler, more interpretable trees, selects relevant covariates, and identifies nested opposite effects while ensuring robust estimation of causal effects when causally admissible variables are provided. Our approach addresses the limitations of traditional causal inference and machine learning methods by introducing an interpretable framework that supports non-expert practitioners while explicitly acknowledging causal assumptions and scope limitations, enabling more reliable and informed decision-making in complex observational data environments.

Related papers

• Scalable Contrastive Causal Discovery under Unknown Soft Interventions [3.165716101116899]
  We propose a scalable causal discovery model for paired observational and interventional settings with shared underlying causal structure and unknown soft interventions.<n> Experiments on synthetic data demonstrate improved causal structure recovery, generalization to unseen graphs with held-out causal mechanisms, and scalability to larger graphs.
  arXiv  Detail & Related papers  (2026-03-03T18:16:16Z)
• Causal Discovery with Mixed Latent Confounding via Precision Decomposition [0.0]
  Differentiable and score-based DAG learners can misinterpret global latent effects as causal edges, while latent-variable graphical models recover only undirected structure.<n>We propose textscDCL-DECOR, a modular, precision-led pipeline that separates these roles.
  arXiv  Detail & Related papers  (2025-12-31T08:03:41Z)
• The Curse of CoT: On the Limitations of Chain-of-Thought in In-Context Learning [56.574829311863446]
  Chain-of-Thought (CoT) prompting has been widely recognized for its ability to enhance reasoning capabilities in large language models (LLMs)<n>We demonstrate that CoT and its reasoning variants consistently underperform direct answering across varying model scales and benchmark complexities.<n>Our analysis uncovers a fundamental hybrid mechanism of explicit-implicit reasoning driving CoT's performance in pattern-based ICL.
  arXiv  Detail & Related papers  (2025-04-07T13:51:06Z)
• Addressing pitfalls in implicit unobserved confounding synthesis using explicit block hierarchical ancestral sampling [1.7037247867649157]
  We show that state-of-the-art protocols have two distinct issues that hinder unbiased sampling from the complete space of causal models.<n>We propose an improved explicit modeling approach for unobserved confounding, leveraging block-hierarchical ancestral generation of ground truth causal graphs.
  arXiv  Detail & Related papers  (2025-03-12T09:38:40Z)
• VISPUR: Visual Aids for Identifying and Interpreting Spurious Associations in Data-Driven Decisions [8.594140167290098]
  Simpson's paradox is a phenomenon where aggregated and subgroup-level associations contradict with each other. Existing tools provide little insights for humans to locate, reason about, and prevent pitfalls of spurious association in practice. We propose VISPUR, a visual analytic system that provides a causal analysis framework and a human-centric workflow for tackling spurious associations.
  arXiv  Detail & Related papers  (2023-07-26T18:40:07Z)
• Advancing Counterfactual Inference through Nonlinear Quantile Regression [77.28323341329461]
  We propose a framework for efficient and effective counterfactual inference implemented with neural networks. The proposed approach enhances the capacity to generalize estimated counterfactual outcomes to unseen data. Empirical results conducted on multiple datasets offer compelling support for our theoretical assertions.
  arXiv  Detail & Related papers  (2023-06-09T08:30:51Z)
• Invariant Causal Set Covering Machines [48.169632766444906]
  Rule-based models, such as decision trees, appeal to practitioners due to their interpretable nature.<n>However, the learning algorithms that produce such models are often vulnerable to spurious associations and thus, they are not guaranteed to extract causally-relevant insights.<n>We propose Invariant Causal Set Covering Machines, an extension of the classical Set Covering Machine algorithm for conjunctions/disjunctions of binary-valued rules that provably avoids spurious associations.
  arXiv  Detail & Related papers  (2023-06-07T20:52:01Z)
• Disentangling Observed Causal Effects from Latent Confounders using Method of Moments [67.27068846108047]
  We provide guarantees on identifiability and learnability under mild assumptions. We develop efficient algorithms based on coupled tensor decomposition with linear constraints to obtain scalable and guaranteed solutions.
  arXiv  Detail & Related papers  (2021-01-17T07:48:45Z)
• Structural Causal Models Are (Solvable by) Credal Networks [70.45873402967297]
  Causal inferences can be obtained by standard algorithms for the updating of credal nets. This contribution should be regarded as a systematic approach to represent structural causal models by credal networks. Experiments show that approximate algorithms for credal networks can immediately be used to do causal inference in real-size problems.
  arXiv  Detail & Related papers  (2020-08-02T11:19:36Z)
• A Critical View of the Structural Causal Model [89.43277111586258]
  We show that one can identify the cause and the effect without considering their interaction at all. We propose a new adversarial training method that mimics the disentangled structure of the causal model. Our multidimensional method outperforms the literature methods on both synthetic and real world datasets.
  arXiv  Detail & Related papers  (2020-02-23T22:52:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.