Reinforcement Learning Foundation Models Should Already Be A Thing

• URL: http://arxiv.org/abs/2606.18812v2
• Date: Thu, 18 Jun 2026 16:24:52 GMT
• Title: Reinforcement Learning Foundation Models Should Already Be A Thing
• Abstract summary: Foundation models for language and vision are powered by internet-scale data, while structured domains are powered by synthetic data.<n>This substitute shifts the challenge from collection to prior design.<n>We train a Graph Attention Network entirely on synthetic MDPs and show that, with no task-specific tuning, it solves held-out benchmarks in context.
• Score: 0.4640835690336653
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Foundation models for language and vision are powered by internet-scale data, while structured domains such as tabular prediction are powered by synthetic data. This substitute shifts the challenge from collection to prior design. Such priors already exist for many structured tasks: TabPFN and its successors solve tabular classification with a transformer pretrained on a synthetic Bayesian prior. We make two points. \textbf{First}, reinforcement learning is the conspicuous gap: sampling a synthetic MDP is as feasible as sampling a synthetic tabular dataset, yet no in-context RL work treats prior design as a primary objective. \textbf{Second}, MDPs admit a fixed-size sufficient statistic, independent of the episodes observed and tabular in shape, which makes them directly amenable to the attention-based architectures used for tabular foundation models, with a policy head replacing the supervised target. Together these define the agenda for an RL foundation model. As a proof of concept, we train a Graph Attention Network entirely on synthetic MDPs and show that, with no task-specific tuning, it solves held-out tabular benchmarks in context, both online and offline: online, in far fewer episodes than UCB-VI and tabular Q-learning, and offline, competitively with VI-LCB.

Related papers

• Beyond IID: How General Are Tabular Foundation Models, Really? [76.03669797389378]
  Research communities are increasingly evaluating foundation models on diverse datasets and tasks.<n>Standard benchmarks are mostly defined for tasks where tabular foundation models already excel.<n>We introduce BeyondArena, the first unified holistic benchmark for tabular data.
  arXiv  Detail & Related papers  (2026-06-29T14:55:52Z)
• Tabular Foundation Model for Generative Modelling [34.3599321018728]
  Generative modelling requires robust, holistic representation learning for a given data modality, rather than optimisation for a supervised prediction target alone.<n>Existing tabular foundation generators have not yet consistently matched strong dataset-specific generators in synthetic data quality.<n>We introduce TabFORGE, built on pretrained textbfTabular textbfFOundational textbfRepresentations for textbfGEneration.
  arXiv  Detail & Related papers  (2026-05-10T08:52:28Z)
• Relational In-Context Learning via Synthetic Pre-training with Structural Prior [60.404256960057545]
  RDB-PFN is the first relational foundation model trained purely via $textbfsynthetic$.<n>Inspired by Prior-Data Fitted Networks (PFNs) where synthetic data generated from Structural Causal Models (SCMs) enables reasoning on single tables.<n>Experiments verify RDB-PFN achieves strong few-shot performance on 19 real-world prediction tasks.
  arXiv  Detail & Related papers  (2026-03-04T07:30:54Z)
• PluRel: Synthetic Data unlocks Scaling Laws for Relational Foundation Models [51.42043158297229]
  We introduce Pluel, a framework to synthesize multi-tabular relational databases from scratch.<n>In a step-by-step fashion, Pluel models (1) schemas with directed graphs, (2) inter-table primary-foreign key connectivity with bipartite graphs, and, (3) feature distributions in tables via conditional causal mechanisms.
  arXiv  Detail & Related papers  (2026-02-03T21:35:18Z)
• Mitra: Mixed Synthetic Priors for Enhancing Tabular Foundation Models [85.64873567417396]
  We introduce Mitra, a TFM trained on a curated mixture of synthetic priors selected for their diversity, distinctiveness, and performance on real-world data.<n>Mitra consistently outperforms state-of-the-art TFMs, such as TabPFNv2 and TabICL, across both classification and regression benchmarks.
  arXiv  Detail & Related papers  (2025-10-24T07:15:06Z)
• A Pre-training Framework for Relational Data with Information-theoretic Principles [57.93973948947743]
  We introduce Task Vector Estimation (TVE), a novel pre-training framework that constructs supervisory signals via set-based aggregation over relational graphs.<n>TVE consistently outperforms traditional pre-training baselines.<n>Our findings advocate for pre-training objectives that encode task heterogeneity and temporal structure as design principles for predictive modeling on relational databases.
  arXiv  Detail & Related papers  (2025-07-14T00:17:21Z)
• TabDPT: Scaling Tabular Foundation Models on Real Data [20.00390825519329]
  We propose an approach to combine ICL-based retrieval with self supervised learning to train foundation models.<n>We show that incorporating real data during the pre-training phase can lead to significantly faster training and better generalization to unseen data.<n>Our resulting model, TabDPT, achieves top performance on both regression (CTR23) and classification (CC18) benchmarks.
  arXiv  Detail & Related papers  (2024-10-23T18:00:00Z)
• Scaling TabPFN: Sketching and Feature Selection for Tabular Prior-Data Fitted Networks [31.82225213006849]
  Tabular classification has traditionally relied on supervised algorithms, which estimate the parameters of a prediction model using its training data. Recently, Prior-Data Fitted Networks (PFNs) such as TabPFN have successfully learned to classify tabular data in-context. While such models show great promise, their applicability to real-world data remains limited due to the computational scale needed.
  arXiv  Detail & Related papers  (2023-11-17T16:04:27Z)
• FLIP: Fine-grained Alignment between ID-based Models and Pretrained Language Models for CTR Prediction [49.510163437116645]
  Click-through rate (CTR) prediction plays as a core function module in personalized online services. Traditional ID-based models for CTR prediction take as inputs the one-hot encoded ID features of tabular modality. Pretrained Language Models(PLMs) has given rise to another paradigm, which takes as inputs the sentences of textual modality. We propose to conduct Fine-grained feature-level ALignment between ID-based Models and Pretrained Language Models(FLIP) for CTR prediction.
  arXiv  Detail & Related papers  (2023-10-30T11:25:03Z)
• Towards Cross-Table Masked Pretraining for Web Data Mining [22.952238405240188]
  We propose an innovative, generic, and efficient cross-table pretraining framework, dubbed as CM2. Our experiments demonstrate CM2's state-of-the-art performance and validate that cross-table pretraining can enhance various downstream tasks.
  arXiv  Detail & Related papers  (2023-07-10T02:27:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.