TS-Fault: Benchmarking Time Series Forecasters Against Structural Faults

• URL: http://arxiv.org/abs/2606.18539v1
• Date: Tue, 16 Jun 2026 23:15:20 GMT
• Title: TS-Fault: Benchmarking Time Series Forecasters Against Structural Faults
• Abstract summary: Time series forecasting (TSF) underpins consequential decisions in energy, transportation, finance, and healthcare.<n>TSF models are almost universally ranked by a single number on clean held-out data, under the implicit assumption that it predicts deployed reliability.<n>We present TS-Fault, a benchmark that evaluates forecasting models under explicit, parameterized fault scenarios with controllable semantic difficulty.
• Score: 22.88447833683736
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Time series forecasting (TSF) underpins consequential decisions in energy, transportation, finance, and healthcare, yet TSF models are almost universally ranked by a single number (e.g., average error) on clean held-out data, under the implicit assumption that it predicts deployed reliability. However, real faults are not i.i.d noise but structured events with temporal shape, broken cross-variable dependencies, regime change coupled with missingness, and causal propagation across a sensing pipeline. Treating TSF robustness as a data-quality problem, we present TS-Fault, a benchmark that evaluates forecasting models under explicit, parameterized fault scenarios with controllable semantic difficulty. TS-Fault organizes recurring failures into four modes along two orthogonal axes (observation- vs mechanism-level; univariate vs multivariate) and injects each fault into the most prediction-critical window via a unified importance score. This design enables robustness to be tested against the structures models actually rely on, rather than reduced to generic noise sensitivity. We evaluate 21 models across 6 datasets, 4 modes, and 5 difficulty levels under a paired clean/corrupt protocol. The results reveal three findings that contradict common leaderboard intuition: (i) clean-data accuracy anti-correlates with robustness; (ii) clean rankings are preserved under observation-level faults but reshuffled under mechanism-level faults; and (iii) all catastrophic failures occur under mechanism-level faults, with foundation models achieving the highest clean-data accuracy yet exhibiting the greatest fragility. The code is publicly available at https://github.com/Ray-zyy/TS-Fault.

Related papers

• Conformal Changepoint Localization and Root Cause Analysis with Corrupted Observations [55.76952683833966]
  This paper proposes weighted CONCH (W-CONCH) and weighted CROC (W-CROC) to reduce confidence set size when data may be corrupted.<n>Experiments on image-based and real-world changepoint and root-cause benchmarks show that uncertainty-based weighting substantially reduces confidence set size.
  arXiv  Detail & Related papers  (2026-07-29T05:16:59Z)
• Fortress: A Case Study in Stabilizing Search Recommendations via Temporal Data Augmentation and Feature Pruning [2.6494136490021387]
  Fortress identifies and prunes features that contribute to inconsistent prediction scores over time.<n>We validate Fortress on a query-to-app relevance model in a large-scale app marketplace.
  arXiv  Detail & Related papers  (2026-05-14T18:13:05Z)
• Benchmarking Sensor-Fault Robustness in Forecasting [34.25988781693566]
  We introduce SensorFault-Bench, a shared CPS-grounded sensor-fault stress-test protocol for evaluating forecasting architectures and robustness-improvement methods.<n>It reports worst-scenario degradation, clean mean squared error (MSE), and worst-scenario fault-time MSE, separating relative robustness from absolute error.<n> SensorFault-Bench provides open-source code, documented data access, and reproduction and extension guides.
  arXiv  Detail & Related papers  (2026-05-11T16:41:14Z)
• LiveFMBench: Unveiling the Power and Limits of Agentic Workflows in Specification Generation [75.05397479715576]
  Large language models (LLMs) and agents have shown promising progress, but their true capabilities and failure modes remain unclear.<n>We present the first systematic and contamination-aware study of LLM- and agent-based formal specification generation for C programs.
  arXiv  Detail & Related papers  (2026-05-02T11:31:33Z)
• Same Answer, Different Representations: Hidden instability in VLMs [65.36933543377346]
  We introduce a representation-aware and frequency-aware evaluation framework that measures internal embedding drift, spectral sensitivity, and structural smoothness.<n>We apply this framework to modern Vision Language Models (VLMs) across the SEEDBench, MMMU, and POPE datasets.
  arXiv  Detail & Related papers  (2026-02-06T12:24:26Z)
• Noisy Analysis of Quantum SMOTE on Condition Monitoring and Fault Classification in Industrial and Energy Systems [0.5505634045241289]
  Imbalanced machine learning models are a fundamental issue in industrial condition monitoring and fault classification pipelines.<n>This work presents a detailed benchmarking and investigation of classical classifiers under class imbalance mitigation.<n>The results show that QSMOTE consistently corrects distributional skew and significantly enhances the performance of non-linear classifiers.
  arXiv  Detail & Related papers  (2026-01-16T16:44:38Z)
• Benchmarking Corruption Robustness of LVLMs: A Discriminative Benchmark and Robustness Alignment Metric [49.393713730706445]
  We introduce Bench-C, a benchmark emphasizing discriminative samples for assessing corruption robustness.<n>We propose the Robustness Alignment Score (RAS), a unified metric that measures degradation in logit-level prediction structure.
  arXiv  Detail & Related papers  (2025-11-24T12:07:56Z)
• Revisiting Multivariate Time Series Forecasting with Missing Values [65.30332997607141]
  Missing values are common in real-world time series.<n>Current approaches have developed an imputation-then-prediction framework that uses imputation modules to fill in missing values, followed by forecasting on the imputed data.<n>This framework overlooks a critical issue: there is no ground truth for the missing values, making the imputation process susceptible to errors that can degrade prediction accuracy.<n>We introduce Consistency-Regularized Information Bottleneck (CRIB), a novel framework built on the Information Bottleneck principle.
  arXiv  Detail & Related papers  (2025-09-27T20:57:48Z)
• BayesTTA: Continual-Temporal Test-Time Adaptation for Vision-Language Models via Gaussian Discriminant Analysis [41.09181390655176]
  Vision-language models (VLMs) such as CLIP achieve strong zero-shot recognition but degrade significantly under textittemporally evolving distribution shifts common in real-world scenarios.<n>We formalize this practical problem as textitContinual-Temporal Test-Time Adaptation (CT-TTA), where test distributions evolve gradually over time.<n>We propose textitBayesTTA, a Bayesian adaptation framework that enforces temporally consistent predictions and dynamically aligns visual representations.
  arXiv  Detail & Related papers  (2025-07-11T14:02:54Z)
• When Does Confidence-Based Cascade Deferral Suffice? [69.28314307469381]
  Cascades are a classical strategy to enable inference cost to vary adaptively across samples. A deferral rule determines whether to invoke the next classifier in the sequence, or to terminate prediction. Despite being oblivious to the structure of the cascade, confidence-based deferral often works remarkably well in practice.
  arXiv  Detail & Related papers  (2023-07-06T04:13:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.