When Tabular Foundation Models Meet Strategic Tabular Data: A Prior Alignment Approach

• URL: http://arxiv.org/abs/2605.19662v1
• Date: Tue, 19 May 2026 10:55:35 GMT
• Title: When Tabular Foundation Models Meet Strategic Tabular Data: A Prior Alignment Approach
• Abstract summary: Tabular foundation models based on pretrained prior-data fitted networks(PFNs)<n>We show that strategic manipulation creates a mismatch between the non-strategic prior learned during pretraining and the post-manipulation strategic prior.<n>We propose textbfStrategic Prior-data Fitted Networktextit(SPN), an inference-time strategy-aware framework.
• Score: 80.24372450818323
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Tabular foundation models based on pretrained prior-data fitted networks~(PFNs) have shown strong generalization on diverse tabular tasks, but they are typically designed for \emph{non-strategic} settings where data distributions are independent of deployed classifiers. In many real-world decision scenarios, however, individuals may strategically modify their features after deployment to obtain favorable outcomes, inducing a post-deployment distribution shift. This paper studies whether PFN-style tabular foundation models can generalize to such \emph{strategic} tabular data. We show that strategic manipulation creates a mismatch between the non-strategic prior learned during pretraining and the post-manipulation strategic prior, which leads to systematic prediction bias. To address this issue, we propose \textbf{Strategic Prior-data Fitted Network}~\textit{(SPN)}, an inference-time strategy-aware framework that adapts tabular foundation models to strategic environments without retraining. SPN constructs strategic in-context examples to approximate post-manipulation inputs and aligns PFN predictions with the induced strategic distribution. Experiments on real-world and synthetic tabular datasets show that SPN consistently improves robustness and predictive performance under strategic manipulation compared with both tabular foundation models and classical tabular methods.

Related papers

• Reinforcement Learning Foundation Models Should Already Be A Thing [0.4640835690336653]
  Foundation models for language and vision are powered by internet-scale data, while structured domains are powered by synthetic data.<n>This substitute shifts the challenge from collection to prior design.<n>We train a Graph Attention Network entirely on synthetic MDPs and show that, with no task-specific tuning, it solves held-out benchmarks in context.
  arXiv  Detail & Related papers  (2026-06-17T08:27:27Z)
• Tabular Foundation Model for Generative Modelling [34.3599321018728]
  Generative modelling requires robust, holistic representation learning for a given data modality, rather than optimisation for a supervised prediction target alone.<n>Existing tabular foundation generators have not yet consistently matched strong dataset-specific generators in synthetic data quality.<n>We introduce TabFORGE, built on pretrained textbfTabular textbfFOundational textbfRepresentations for textbfGEneration.
  arXiv  Detail & Related papers  (2026-05-10T08:52:28Z)
• Mitra: Mixed Synthetic Priors for Enhancing Tabular Foundation Models [85.64873567417396]
  We introduce Mitra, a TFM trained on a curated mixture of synthetic priors selected for their diversity, distinctiveness, and performance on real-world data.<n>Mitra consistently outperforms state-of-the-art TFMs, such as TabPFNv2 and TabICL, across both classification and regression benchmarks.
  arXiv  Detail & Related papers  (2025-10-24T07:15:06Z)
• Self-Rewarding PPO: Aligning Large Language Models with Demonstrations Only [70.43369087819332]
  Supervised fine-tuning (SFT) has emerged as a crucial method for aligning large language models with human-annotated demonstrations.<n>We propose Self-Rewarding PPO, a novel fine-tuning method that leverages on-policy techniques to enhance generalization performance.
  arXiv  Detail & Related papers  (2025-10-24T02:02:13Z)
• Plan before Solving: Problem-Aware Strategy Routing for Mathematical Reasoning with LLMs [49.995906301946]
  Existing methods usually leverage a fixed strategy to guide Large Language Models (LLMs) to perform mathematical reasoning.<n>Our analysis reveals that the single strategy cannot adapt to problem-specific requirements and thus overlooks the trade-off between effectiveness and efficiency.<n>We propose Planning and Routing through Instance-Specific Modeling (PRISM), a novel framework that decouples mathematical reasoning into two stages: strategy planning and targeted execution.
  arXiv  Detail & Related papers  (2025-09-29T07:22:41Z)
• In-Context Learning Strategies Emerge Rationally [36.21854069812912]
  We show that when trained to learn a mixture of tasks, strategies learned by a model for performing ICL can be captured by a family of Bayesian predictors.<n>We develop a hierarchical Bayesian framework that almost perfectly predicts Transformer next-token predictions throughout training.<n>Our work advances an explanatory and predictive account of ICL grounded in tradeoffs between strategy loss and complexity.
  arXiv  Detail & Related papers  (2025-06-21T23:49:08Z)
• Make Still Further Progress: Chain of Thoughts for Tabular Data Leaderboard [27.224577475861214]
  Tabular data, a fundamental data format in machine learning, is predominantly utilized in competitions and real-world applications.<n>We propose an in-context ensemble framework for tabular prediction that leverages large language models.<n>Our method constructs a context around each test instance using its nearest neighbors and the predictions from a pool of external models.
  arXiv  Detail & Related papers  (2025-05-19T17:52:58Z)
• Step-wise Distribution Alignment Guided Style Prompt Tuning for Source-free Cross-domain Few-shot Learning [53.77707279483278]
  Cross-domain few-shot learning methods face challenges with large-scale pre-trained models due to inaccessible source data and training strategies.<n>This paper introduces Step-wise Distribution Alignment Guided Style Prompt Tuning (StepSPT)<n>StepSPT implicitly narrows domain gaps through prediction distribution optimization.
  arXiv  Detail & Related papers  (2024-11-15T09:34:07Z)
• Understanding Robust Generalization in Learning Regular Languages [85.95124524975202]
  We study robust generalization in the context of using recurrent neural networks to learn regular languages. We propose a compositional strategy to address this. We theoretically prove that the compositional strategy generalizes significantly better than the end-to-end strategy.
  arXiv  Detail & Related papers  (2022-02-20T02:50:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.