Flow-Corrected Thompson Sampling for Non-Stationary Contextual Bandits

• URL: http://arxiv.org/abs/2606.23933v1
• Date: Mon, 22 Jun 2026 20:52:19 GMT
• Title: Flow-Corrected Thompson Sampling for Non-Stationary Contextual Bandits
• Abstract summary: We study non-stationary linear contextual bandits where the reward model drifts over time.<n>We propose Flow-Corrected Thompson Sampling (fcTS), a Bayesian method that reuses experience by transporting past rewards to the present.
• Score: 3.2848713528308817
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study non-stationary linear contextual bandits where the reward model drifts over time, rendering classical contextual bandit algorithms brittle because historical data becomes systematically biased. We propose Flow-Corrected Thompson Sampling (fcTS), a Bayesian method that reuses experience by transporting past rewards to the present using an explicit drift model and incorporating each transported observation with a confidence weight that reflects transport reliability. This yields a unified template that specializes in (i) linear parameter drift via online slope estimation and reward correction, (ii) periodic variation via phase-aware reuse across cycles, and (iii) recurring regime switches via changepoint detection and regime-specific posterior memory. The resulting posterior updates remain closed-form under a linear Gaussian model and can be implemented efficiently with truncated, incrementally updated sufficient statistics. Across five controlled case studies and a semi-synthetic portfolio-selection benchmark with multiple overlapping non-stationarities, fcTS outperforms standard forgetting-based baselines (discounting, sliding windows, and periodic restarts), with the largest gains in settings exhibiting recurring temporal structure. These results demonstrate that when non-stationarity is structured, correcting and reweighting historical observations can be substantially more sample-efficient than uniformly discarding them.

Related papers

• Continual Test-Time Adaptation via Entropy Sensitivity-Guidance in Strict Online Setting [4.775158008378889]
  We introduce Sensitivity-Guided Erasing Adaptation (SEGA), a method for strict online continual TTA (CTTA) on corruption-style streams.<n>SEGA uses a small number of structured erasures to probe how predictive entropy changes as information is removed.<n>In experiments on ImageNet-C, CIFAR10/100-C, and corruption-generated aquaculture streams treated as controlled corruption-style proxies, SEGA yields consistent robustness and stability gains.
  arXiv  Detail & Related papers  (2026-08-30T17:30:51Z)
• GatedLinear: Adaptive Routing of Complementary Linear Bases for Time Series Forecasting [11.155508081323694]
  Time series forecasting requires models to capture diverse, often mutually exclusive, temporal dynamics.<n>Recent deep learning models have improved accuracy, but they typically force these diverse patterns through a single computational backbone.<n>We propose GatedLinear: a lightweight framework that frames forecasting as the adaptive routing of complementary linear bases.
  arXiv  Detail & Related papers  (2026-07-10T15:42:01Z)
• RBE-Flow: Recurrent Bayesian Estimation on Feature Manifolds for Cross-Modal Registration [14.194086645453767]
  Cross-pixel image registration is challenging due to severe non-linear radiometric and uncertainty.<n>In this paper we propose a novel framework that reformulates dense-modal flow estimation.
  arXiv  Detail & Related papers  (2026-06-29T15:58:25Z)
• Simple Approximation and Derivative Free Inference-Time Scaling for Diffusion Models via Sequential Monte Carlo on Path Measures [13.41132148962756]
  textttURGE is a derivative-free inference-time scaling algorithm that performs path-wise importance reweighting via a change of measure.<n>No score, no Hessian, and no PDE evaluation is required. We establish an equivalence between path-wise and particle-wise SMC.
  arXiv  Detail & Related papers  (2026-05-18T04:45:33Z)
• RAVEN: Real-time Autoregressive Video Extrapolation with Consistency-model GRPO [53.38929612273108]
  Causal autoregressive video diffusion models support real-time streaming generation by extrapolating future chunks from previously generated content.<n>We introduce the Real-time Autoregressive Video Extrapolation Network (RAVEN), a training-time test framework that repacks each self rollout into an interleaved sequence of clean historical endpoints.
  arXiv  Detail & Related papers  (2026-05-14T17:59:30Z)
• Temporal Structure Matters for Efficient Test-Time Adaptation in Wearable Human Activity Recognition [0.20391237204597365]
  Test-time adaptation (TTA) mitigates performance degradation under real-world cross-user distribution shifts.<n>Existing methods largely inherit assumptions from vision tasks and underexploit the inherent inter-window temporal structure in WHAR streams.<n>We propose SIGHT, a lightweight and backpropagation-free TTA framework for WHAR, enabling real-time edge deployment.
  arXiv  Detail & Related papers  (2026-05-06T08:06:27Z)
• Invariance-Based Dynamic Regret Minimization [8.349786817840858]
  We consider non-stationary linear bandits where the linear parameter connecting contexts to the reward changes over time.<n>We propose to leverage historical data while adapting to changes by assuming the reward model decomposes into stationary and non-stationary components.
  arXiv  Detail & Related papers  (2026-03-04T08:47:02Z)
• Learnable Chernoff Baselines for Inference-Time Alignment [64.81256817158851]
  We introduce Learnable Chernoff Baselines as a method for efficiently and approximately sampling from exponentially tilted kernels.<n>We establish total-variation guarantees to the ideal aligned model, and demonstrate in both continuous and discrete diffusion settings that LCB sampling closely matches ideal rejection sampling.
  arXiv  Detail & Related papers  (2026-02-08T00:09:40Z)
• Temporal Pair Consistency for Variance-Reduced Flow Matching [13.328987133593154]
  Temporal Pair Consistency (TPC) is a lightweight variance-reduction principle that couples velocity predictions at paired timesteps along the same probability path.<n>Instantiated within flow matching, TPC improves sample quality and efficiency across CIFAR-10 and ImageNet at multiple resolutions.
  arXiv  Detail & Related papers  (2026-02-04T00:05:21Z)
• LARA: A Light and Anti-overfitting Retraining Approach for Unsupervised Time Series Anomaly Detection [49.52429991848581]
  We propose a Light and Anti-overfitting Retraining Approach (LARA) for deep variational auto-encoder based time series anomaly detection methods (VAEs) This work aims to make three novel contributions: 1) the retraining process is formulated as a convex problem and can converge at a fast rate as well as prevent overfitting; 2) designing a ruminate block, which leverages the historical data without the need to store them; and 3) mathematically proving that when fine-tuning the latent vector and reconstructed data, the linear formations can achieve the least adjusting errors between the ground truths and the fine-tuned ones.
  arXiv  Detail & Related papers  (2023-10-09T12:36:16Z)
• Kalman Filter for Online Classification of Non-Stationary Data [101.26838049872651]
  In Online Continual Learning (OCL) a learning system receives a stream of data and sequentially performs prediction and training steps. We introduce a probabilistic Bayesian online learning model by using a neural representation and a state space model over the linear predictor weights. In experiments in multi-class classification we demonstrate the predictive ability of the model and its flexibility to capture non-stationarity.
  arXiv  Detail & Related papers  (2023-06-14T11:41:42Z)
• Debiased Fine-Tuning for Vision-language Models by Prompt Regularization [56.48290708901531]
  We present a new paradigm for fine-tuning large-scale vision pre-trained models on downstream task, dubbed Prompt Regularization (ProReg)<n>ProReg uses the prediction by prompting the pretrained model to regularize the fine-tuning.<n>We show the consistently strong performance of ProReg compared with conventional fine-tuning, zero-shot prompt, prompt tuning, and other state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-01-29T11:53:55Z)
• Anomaly Detection of Time Series with Smoothness-Inducing Sequential Variational Auto-Encoder [59.69303945834122]
  We present a Smoothness-Inducing Sequential Variational Auto-Encoder (SISVAE) model for robust estimation and anomaly detection of time series. Our model parameterizes mean and variance for each time-stamp with flexible neural networks. We show the effectiveness of our model on both synthetic datasets and public real-world benchmarks.
  arXiv  Detail & Related papers  (2021-02-02T06:15:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.