When the Martingale Never Stops Firing: Anytime-Valid Gating on Real Forecast Streams

• URL: http://arxiv.org/abs/2608.30502v1
• Date: Mon, 31 Aug 2026 09:32:11 GMT
• Title: When the Martingale Never Stops Firing: Anytime-Valid Gating on Real Forecast Streams
• Abstract summary: Anytime-valid inference promises evidence that can be acted on at any moment.<n>A deployment inherits it only if the stream it monitors behaves exchangeably.
• Score: 0.614481021961242
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine learning systems are increasingly corrected while they run, and the decision of when to intervene is increasingly delegated to statistical monitors. Anytime-valid inference promises evidence that can be acted on at any moment, exactly the guarantee this setting needs, and it is moving from theory into deployed monitoring. Conformal test martingales are the change-detection instrument, and Ville's inequality caps their false-alarm probability on exchangeable data. The guarantee is conditional. A deployment inherits it only if the stream it monitors behaves exchangeably. The premise is hardest to satisfy where these monitors are most useful, on dependent data and inside loops where the monitor modifies the learner whose scores it reads. It is also rarely measured. We measure it in a pre-specified case study, where such a monitor gates the online updates of a Kalman adapter correcting frozen time-series foundation models on five forecasting streams. On exchangeable synthetic streams, the same implementation fires in at most 1 of 60 runs. On the real streams, at alpha = 0.05, 135 of 135 clean-stream runs fired. The construction does not explain the firing; the failure comes from the deployed score stream itself. Repeated fires hold the gate's drift response active, and the gated filter amplifies the very transient it was designed to prevent. The component worth keeping makes no validity claim. Huber-style gating of the filter's own updates cuts isolated-spike degradation by an order of magnitude with no dataset specific tuning. Anytime-valid methods proposed for dependent data should therefore be accompanied by null-calibration controls and mechanism traces.

Related papers

• Underwater Dead Reckoning with Deployable Situation-Triggered Covariance Scheduling [3.2548794659022398]
  This paper presents the Situation-Triggered Calibrated Adaptive Robust Extended Kalman Filter for a BlueROV2.<n>An onboard probabilistic trigger identifies the current motion situation while one error-state filter runs continuously.<n>When the trigger is confident, the filter changes only to the corresponding pre-calibrated process- and measurement-noise matrices.
  arXiv  Detail & Related papers  (2026-07-12T06:27:57Z)
• Bistable by Construction: Wall-Clock-Calibrated State Monitors Have No Moment-Detection Regime at Agent Cadence [0.0]
  We report a State Saturation Trap: threshold-on-state triggers over a continuous affect engine become near-constant alarms on SWE-bench agents.<n>We correct the record (erratum, v2) and treat the flaw as an experiment.<n>We conclude that wall-clock-calibrated leaky-integrator monitors admit no regime in which they act as moment detectors on agent streams.
  arXiv  Detail & Related papers  (2026-06-15T21:25:52Z)
• Don't Pause! Every prediction matters in a streaming video [55.509551643600794]
  We introduce SPOT-Bench, featuring multi-turn proactive queries that evaluate general streaming perception and assistive capabilities.<n> SPOT-Bench comes with Timeliness-F1, a metric that measures streaming predictions by their temporal precision and balanced coverage across the entire video.<n>Our benchmark reveals: (i) offline models detect events reliably but spam predictions unprompted; (ii) post-training for silence reduces spamming but induces unresponsiveness; (iii) half of the streaming video expects no response, which we term dead-time - compute spent here does not affect response latency.
  arXiv  Detail & Related papers  (2026-04-27T11:07:03Z)
• When Your Model Stops Working: Anytime-Valid Calibration Monitoring [0.0]
  PITMonitor is an anytime-valid calibration-specific monitor that detects distributional shifts in probability integral transforms.<n>On river's FriedmanDrift benchmark, PITMonitor detection rates achieves competitive with the strongest baselines.
  arXiv  Detail & Related papers  (2026-03-13T16:50:14Z)
• Revisiting Multivariate Time Series Forecasting with Missing Values [65.30332997607141]
  Missing values are common in real-world time series.<n>Current approaches have developed an imputation-then-prediction framework that uses imputation modules to fill in missing values, followed by forecasting on the imputed data.<n>This framework overlooks a critical issue: there is no ground truth for the missing values, making the imputation process susceptible to errors that can degrade prediction accuracy.<n>We introduce Consistency-Regularized Information Bottleneck (CRIB), a novel framework built on the Information Bottleneck principle.
  arXiv  Detail & Related papers  (2025-09-27T20:57:48Z)
• WATCH: Adaptive Monitoring for AI Deployments via Weighted-Conformal Martingales [22.789611187514975]
  Methods for nonparametric sequential testing -- especially conformal test martingales (CTMs) and anytime-valid inference -- offer promising tools for this monitoring task.<n>Existing approaches are restricted to monitoring limited hypothesis classes or alarm criteria''
  arXiv  Detail & Related papers  (2025-05-07T17:53:47Z)
• Streaming Motion Forecasting for Autonomous Driving [71.7468645504988]
  We introduce a benchmark that queries future trajectories on streaming data and we refer to it as "streaming forecasting" Our benchmark inherently captures the disappearance and re-appearance of agents, which is a safety-critical problem yet overlooked by snapshot-based benchmarks. We propose a plug-and-play meta-algorithm called "Predictive Streamer" that can adapt any snapshot-based forecaster into a streaming forecaster.
  arXiv  Detail & Related papers  (2023-10-02T17:13:16Z)
• Sequential Kernelized Independence Testing [77.237958592189]
  We design sequential kernelized independence tests inspired by kernelized dependence measures.<n>We demonstrate the power of our approaches on both simulated and real data.
  arXiv  Detail & Related papers  (2022-12-14T18:08:42Z)
• Tracking the risk of a deployed model and detecting harmful distribution shifts [105.27463615756733]
  In practice, it may make sense to ignore benign shifts, under which the performance of a deployed model does not degrade substantially. We argue that a sensible method for firing off a warning has to both (a) detect harmful shifts while ignoring benign ones, and (b) allow continuous monitoring of model performance without increasing the false alarm rate.
  arXiv  Detail & Related papers  (2021-10-12T17:21:41Z)
• Detecting Rewards Deterioration in Episodic Reinforcement Learning [63.49923393311052]
  In many RL applications, once training ends, it is vital to detect any deterioration in the agent performance as soon as possible. We consider an episodic framework, where the rewards within each episode are not independent, nor identically-distributed, nor Markov. We define the mean-shift in a way corresponding to deterioration of a temporal signal (such as the rewards), and derive a test for this problem with optimal statistical power.
  arXiv  Detail & Related papers  (2020-10-22T12:45:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.