PFN-TS: Thompson Sampling for Contextual Bandits via Prior-Data Fitted Networks

• URL: http://arxiv.org/abs/2605.10137v1
• Date: Mon, 11 May 2026 07:46:10 GMT
• Title: PFN-TS: Thompson Sampling for Contextual Bandits via Prior-Data Fitted Networks
• Abstract summary: We propose PFN-TS, a Thompson sampling algorithm that converts PFN posterior predictives into mean-reward samples.<n>PFN-TS achieves the best average rank across nonlinear synthetic and OpenML classification-to-bandit benchmarks.
• Score: 7.188084723389871
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Thompson sampling is a widely used strategy for contextual bandits: at each round, it samples a reward function from a Bayesian posterior and acts greedily under that sample. Prior-data fitted networks (PFNs), such as TabPFN v2+ and TabICL v2, are attractive candidates for this purpose because they approximate Bayesian posterior predictive distributions in a single forward pass. However, PFNs predict noisy future rewards, while Thompson sampling requires uncertainty over the latent mean reward function. We propose PFN-TS, a Thompson sampling algorithm that converts PFN posterior predictives into mean-reward samples using a subsampled predictive central limit theorem. The method estimates posterior variance from a geometric grid of $O(\log n)$ dataset prefixes rather than the full $O(n)$ predictive sequence used in previous predictive-sequence approaches, and reuses TabICL's cached representations across rounds. We prove consistency of the subsampled variance estimator and give a Bayesian regret bound that decomposes PFN-TS regret into exact posterior-sampling regret under the PFN prior plus approximation terms. Empirically, PFN-TS achieves the best average rank across nonlinear synthetic and OpenML classification-to-bandit benchmarks, remains competitive on linear and BART-generated rewards, and attains the highest estimated policy value in an offline mobile-health evaluation. Code is available at https://anonymous.4open.science/r/PFN_TS-36ED/.

Related papers

• Adaptive Candidate Point Thompson Sampling for High-Dimensional Bayesian Optimization [10.764160559530845]
  We introduce Adaptive Candidate Thompson Sampling (ACTS)<n>ACTS generates candidate points in subspaces guided by the gradient of a surrogate model sample.<n>It produces better samples of maxima and improved optimization across synthetic and real-world benchmarks.
  arXiv  Detail & Related papers  (2026-04-10T02:54:31Z)
• Variance-Aware Feel-Good Thompson Sampling for Contextual Bandits [54.220839560203096]
  We present FGTSVA, a variance-aware Thompson Sampling algorithm for contextual bandits with general reward function with optimal regret bound.<n>With the new decoupling coefficient denoted by $mathrmdc$, FGTS-VA achieves the regret of $tildeO(sqrtmathrmdccdotlog|mathcalF|$, where $|mathcalF|$ is the size of the model space.<n>In the setting of contextual linear bandits, the regret bound of FGTSVA matches that of UCB-based
  arXiv  Detail & Related papers  (2025-11-03T23:25:41Z)
• Thompson Sampling via Fine-Tuning of LLMs [68.1722422968855]
  We propose an alternative based on Thompson sampling that eliminates the need for scalable large acquisition functions.<n>Our approach Thompson Sampling via Finening (ToSFiT) leverages the prior knowledge embedded in prompt-conditioned language models, and adapts incrementally toward the posterior.<n>Our analysis reveals the critical role of careful adaptation to the posterior probability of maximality-a principle that underpins our ToSFiT algorithm.
  arXiv  Detail & Related papers  (2025-10-15T09:13:59Z)
• Efficient Approximate Posterior Sampling with Annealed Langevin Monte Carlo [36.938523047101555]
  We study the problem of posterior sampling in the context of score based generative models.<n>We show that one can tractably sample from a distribution that is simultaneously close to the posterior of a noised prior in KL divergence and the true posterior in Fisher divergence.
  arXiv  Detail & Related papers  (2025-08-11T05:25:24Z)
• An Efficient Rehearsal Scheme for Catastrophic Forgetting Mitigation during Multi-stage Fine-tuning [55.467047686093025]
  A common approach to alleviate such forgetting is to rehearse samples from prior tasks during fine-tuning.<n>We propose a sampling scheme, textttbf mix-cd, that prioritizes rehearsal of collateral damage'' samples.<n>Our approach is computationally efficient, easy to implement, and outperforms several leading continual learning methods in compute-constrained settings.
  arXiv  Detail & Related papers  (2024-02-12T22:32:12Z)
• VITS : Variational Inference Thompson Sampling for contextual bandits [10.028119153832346]
  We introduce and analyze a variant of the Thompson sampling (TS) algorithm for contextual bandits. We propose a new algorithm, Varational Inference Thompson sampling VITS, based on Gaussian Variational Inference. We show that VITS achieves a sub-linear regret bound of the same order in the dimension and number of round as traditional TS for linear contextual bandit.
  arXiv  Detail & Related papers  (2023-07-19T17:53:22Z)
• Langevin Monte Carlo for Contextual Bandits [72.00524614312002]
  Langevin Monte Carlo Thompson Sampling (LMC-TS) is proposed to directly sample from the posterior distribution in contextual bandits. We prove that the proposed algorithm achieves the same sublinear regret bound as the best Thompson sampling algorithms for a special case of contextual bandits.
  arXiv  Detail & Related papers  (2022-06-22T17:58:23Z)
• Non-Probability Sampling Network for Stochastic Human Trajectory Prediction [16.676008193894223]
  Capturing multimodal natures is essential for incorporating pedestrian trajectory prediction. We introduce the Quasi-Carlo method, ensuring uniform coverage on the sampling space, as an alternative to the conventional random sampling. We take an additional step ahead by a learnable sampling network into the existing networks for trajectory prediction.
  arXiv  Detail & Related papers  (2022-03-25T06:41:47Z)
• Transformers Can Do Bayesian Inference [56.99390658880008]
  We present Prior-Data Fitted Networks (PFNs) PFNs leverage in-context learning in large-scale machine learning techniques to approximate a large set of posteriors. We demonstrate that PFNs can near-perfectly mimic Gaussian processes and also enable efficient Bayesian inference for intractable problems.
  arXiv  Detail & Related papers  (2021-12-20T13:07:39Z)
• Analysis and Design of Thompson Sampling for Stochastic Partial Monitoring [91.22679787578438]
  We present a novel Thompson-sampling-based algorithm for partial monitoring. We prove that the new algorithm achieves the logarithmic problem-dependent expected pseudo-regret $mathrmO(log T)$ for a linearized variant of the problem with local observability.
  arXiv  Detail & Related papers  (2020-06-17T05:48:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.