Improving TabPFN's Synthetic Data Generation by Integrating Causal Structure

• URL: http://arxiv.org/abs/2603.10254v1
• Date: Tue, 10 Mar 2026 22:16:44 GMT
• Title: Improving TabPFN's Synthetic Data Generation by Integrating Causal Structure
• Abstract summary: Synthetic data generation addresses data scarcity and privacy constraints.<n>We show how causal structure can be integrated into the generation process.<n>We evaluate these approaches on controlled benchmarks and six CSuite datasets.
• Score: 1.3357838668308473
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Synthetic tabular data generation addresses data scarcity and privacy constraints in a variety of domains. Tabular Prior-Data Fitted Network (TabPFN), a recent foundation model for tabular data, has been shown capable of generating high-quality synthetic tabular data. However, TabPFN is autoregressive: features are generated sequentially by conditioning on the previous ones, depending on the order in which they appear in the input data. We demonstrate that when the feature order conflicts with causal structure, the model produces spurious correlations that impair its ability to generate synthetic data and preserve causal effects. We address this limitation by integrating causal structure into TabPFN's generation process through two complementary approaches: Directed Acyclic Graph (DAG)-aware conditioning, which samples each variable given its causal parents, and a Completed Partially Directed Acyclic Graph (CPDAG)-based strategy for scenarios with partial causal knowledge. We evaluate these approaches on controlled benchmarks and six CSuite datasets, assessing structural fidelity, distributional alignment, privacy preservation, and Average Treatment Effect (ATE) preservation. Across most settings, DAG-aware conditioning improves the quality and stability of synthetic data relative to vanilla TabPFN. The CPDAG-based strategy shows moderate improvements, with effectiveness depending on the number of oriented edges. These results indicate that injecting causal structure into autoregressive generation enhances the reliability of synthetic tabular data.

Related papers

• Differentially Private Synthetic Data via APIs 4: Tabular Data [14.625539540842672]
  State-of-the-art methods often focus on minimizing low-order marginal query errors and overlook the challenges posed by high-order correlations.<n>To address this gap, we extend the Private Evolution framework, originally developed for DP-compliant image and text synthesis.
  arXiv  Detail & Related papers  (2026-06-06T17:08:41Z)
• Generating Logically Consistent Synthetic Supply Chain Data with LLM-Driven Knowledge Graph Reasoning [51.78065387034195]
  This paper introduces textbftextitTabKG, a knowledge-graph-guided framework for logically consistent synthetic supply chain data generation.
  arXiv  Detail & Related papers  (2026-05-26T10:41:37Z)
• Self-Improving Tabular Language Models via Iterative Group Alignment [76.43372236405664]
  We introduce TabGRAA, the first self-improving framework for data generation via automated feedback.<n>It partitions newly generated samples into high- and low-quality groups, then prioritizes a group-relative advantage objective.<n>Experiments show TabGRAA outperforms existing methods in fidelity, utility, and privacy.
  arXiv  Detail & Related papers  (2026-04-21T01:29:52Z)
• RelDiff: Relational Data Generative Modeling with Graph-Based Diffusion Models [83.6013616017646]
  RelDiff is a novel diffusion generative model that synthesizes complete relational databases by explicitly modeling their foreign key graph structure.<n>RelDiff consistently outperforms prior methods in producing realistic and coherent synthetic relational databases.
  arXiv  Detail & Related papers  (2025-05-31T21:01:02Z)
• LLM-TabLogic: Preserving Inter-Column Logical Relationships in Synthetic Tabular Data via Prompt-Guided Latent Diffusion [49.898152180805454]
  Synthetic datasets must maintain domain-specific logical consistency.<n>Existing generative models often overlook these inter-column relationships.<n>This study presents the first method to effectively preserve inter-column relationships without requiring domain knowledge.
  arXiv  Detail & Related papers  (2025-03-04T00:47:52Z)
• Generating Realistic Tabular Data with Large Language Models [49.03536886067729]
  Large language models (LLM) have been used for diverse tasks, but do not capture the correct correlation between the features and the target variable. We propose a LLM-based method with three important improvements to correctly capture the ground-truth feature-class correlation in the real data. Our experiments show that our method significantly outperforms 10 SOTA baselines on 20 datasets in downstream tasks.
  arXiv  Detail & Related papers  (2024-10-29T04:14:32Z)
• FedTabDiff: Federated Learning of Diffusion Probabilistic Models for Synthetic Mixed-Type Tabular Data Generation [5.824064631226058]
  We introduce textitFederated Tabular Diffusion (FedTabDiff) for generating high-fidelity mixed-type tabular data without centralized access to the original datasets. FedTabDiff realizes a decentralized learning scheme that permits multiple entities to collaboratively train a generative model while respecting data privacy and locality. Experimental evaluations on real-world financial and medical datasets attest to the framework's capability to produce synthetic data that maintains high fidelity, utility, privacy, and coverage.
  arXiv  Detail & Related papers  (2024-01-11T21:17:50Z)
• Fake It Till Make It: Federated Learning with Consensus-Oriented Generation [52.82176415223988]
  We propose federated learning with consensus-oriented generation (FedCOG) FedCOG consists of two key components at the client side: complementary data generation and knowledge-distillation-based model training. Experiments on classical and real-world FL datasets show that FedCOG consistently outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-12-10T18:49:59Z)
• Generating tabular datasets under differential privacy [0.0]
  We introduce Differential Privacy (DP) into the training process of deep neural networks. This creates a trade-off between the quality and privacy of the resulting data. We implement novel end-to-end models that leverage attention mechanisms.
  arXiv  Detail & Related papers  (2023-08-28T16:35:43Z)
• CasTGAN: Cascaded Generative Adversarial Network for Realistic Tabular Data Synthesis [0.4999814847776097]
  Generative adversarial networks (GANs) have drawn considerable attention in recent years for their proven capability in generating synthetic data. The validity of the synthetic data and the underlying privacy concerns represent major challenges which are not sufficiently addressed.
  arXiv  Detail & Related papers  (2023-07-01T16:52:18Z)
• Federated Causal Discovery [74.37739054932733]
  This paper develops a gradient-based learning framework named DAG-Shared Federated Causal Discovery (DS-FCD) It can learn the causal graph without directly touching local data and naturally handle the data heterogeneity. Extensive experiments on both synthetic and real-world datasets verify the efficacy of the proposed method.
  arXiv  Detail & Related papers  (2021-12-07T08:04:12Z)
• CTAB-GAN: Effective Table Data Synthesizing [7.336728307626645]
  We develop CTAB-GAN, a conditional table GAN architecture that can model diverse data types. We show that CTAB-GAN remarkably resembles the real data for all three types of variables and results into higher accuracy for five machine learning algorithms, by up 17%.
  arXiv  Detail & Related papers  (2021-02-16T18:53:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.