From Synthetic Priors to Model Behavior: Structural Coverage in Tabular Foundation Models

• URL: http://arxiv.org/abs/2609.06912v1
• Date: Mon, 07 Sep 2026 01:27:45 GMT
• Title: From Synthetic Priors to Model Behavior: Structural Coverage in Tabular Foundation Models
• Abstract summary: We measure how broadly and repeatedly each synthetic prior reaches benchmark tasks using structural coverage and normalized density.<n>We find substantial differences across synthetic pretraining priors.<n>Stronger synthetic-to-benchmark support is generally associated with better relative model performance.
• Score: 15.974042867454365
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Tabular foundation models (TFMs) are commonly pretrained on large collections of procedurally generated synthetic tasks, yet it remains unclear how well these synthetic pretraining priors support the downstream tasks on which the models are evaluated. We study this question from a distribution-level attribution perspective. We recover or reconstruct the synthetic data generators of four TFMs and compare their generated tasks with datasets from two widely used tabular benchmarks. Each dataset is represented by a common set of structural descriptors capturing schema, feature distributions, dependence structure, response properties, and feature--response relationships. In this space, we measure how broadly and repeatedly each synthetic prior reaches benchmark tasks using structural coverage and normalized density, and examine whether stronger local support is associated with better predictive performance. We find substantial differences across synthetic pretraining priors: some generators provide consistently broader and denser support for benchmark tasks than others. Moreover, stronger synthetic-to-benchmark support is generally associated with better relative model performance. These results suggest that structural coverage provides a useful diagnostic for characterizing synthetic pretraining priors and relating their data-generating assumptions to downstream model behavior.

Related papers

• Towards Evaluating Data Priors for Tabular Foundation Models [74.75794086564035]
  We implement a unified interface for publicly available priors and priors constructed from real datasets.<n>We generate training tasks from each prior, train the same model architecture under a fixed training protocol, and evaluate the resulting models on shared downstream classification tasks.<n>Our results show that different priors favor different downstream behaviors, with some achieving stronger absolute performance and others exhibiting more consistent relative rankings across datasets.
  arXiv  Detail & Related papers  (2026-06-28T07:12:45Z)
• Tabular Foundation Model for Generative Modelling [34.3599321018728]
  Generative modelling requires robust, holistic representation learning for a given data modality, rather than optimisation for a supervised prediction target alone.<n>Existing tabular foundation generators have not yet consistently matched strong dataset-specific generators in synthetic data quality.<n>We introduce TabFORGE, built on pretrained textbfTabular textbfFOundational textbfRepresentations for textbfGEneration.
  arXiv  Detail & Related papers  (2026-05-10T08:52:28Z)
• Foundation Priors [0.0]
  Synthetic data reflects both the model's learned patterns and the user's subjective priors, expectations, and biases.<n>We show how synthetic data and the associated foundation prior can be incorporated into standard statistical and econometrics.
  arXiv  Detail & Related papers  (2025-11-30T22:09:37Z)
• Mitra: Mixed Synthetic Priors for Enhancing Tabular Foundation Models [85.64873567417396]
  We introduce Mitra, a TFM trained on a curated mixture of synthetic priors selected for their diversity, distinctiveness, and performance on real-world data.<n>Mitra consistently outperforms state-of-the-art TFMs, such as TabPFNv2 and TabICL, across both classification and regression benchmarks.
  arXiv  Detail & Related papers  (2025-10-24T07:15:06Z)
• RelDiff: Relational Data Generative Modeling with Graph-Based Diffusion Models [83.6013616017646]
  RelDiff is a novel diffusion generative model that synthesizes complete relational databases by explicitly modeling their foreign key graph structure.<n>RelDiff consistently outperforms prior methods in producing realistic and coherent synthetic relational databases.
  arXiv  Detail & Related papers  (2025-05-31T21:01:02Z)
• Scaling Laws of Synthetic Data for Language Models [125.41600201811417]
  We introduce SynthLLM, a scalable framework that transforms pre-training corpora into diverse, high-quality synthetic datasets.<n>Our approach achieves this by automatically extracting and recombining high-level concepts across multiple documents using a graph algorithm.
  arXiv  Detail & Related papers  (2025-03-25T11:07:12Z)
• Approximate learning of parsimonious Bayesian context trees [0.0]
  The proposed framework is tested on synthetic and real-world data examples. It outperforms existing sequence models when fitted to real protein sequences and honeypot computer terminal sessions.
  arXiv  Detail & Related papers  (2024-07-27T11:50:40Z)
• Causality for Tabular Data Synthesis: A High-Order Structure Causal Benchmark Framework [18.11940247961923]
  In this paper, we introduce high-order structural causal information as natural prior knowledge. We propose multiple benchmark tasks, high-order metrics, and causal inference tasks as downstream tasks for evaluating the quality of synthetic data.
  arXiv  Detail & Related papers  (2024-06-12T15:12:49Z)
• SynBench: Task-Agnostic Benchmarking of Pretrained Representations using Synthetic Data [78.21197488065177]
  Recent success in fine-tuning large models, that are pretrained on broad data at scale, on downstream tasks has led to a significant paradigm shift in deep learning. This paper proposes a new task-agnostic framework, textitSynBench, to measure the quality of pretrained representations using synthetic data.
  arXiv  Detail & Related papers  (2022-10-06T15:25:00Z)
• HyperImpute: Generalized Iterative Imputation with Automatic Model Selection [77.86861638371926]
  We propose a generalized iterative imputation framework for adaptively and automatically configuring column-wise models. We provide a concrete implementation with out-of-the-box learners, simulators, and interfaces.
  arXiv  Detail & Related papers  (2022-06-15T19:10:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.