Related papers: TimeSage-EV: A Live Benchmark for Agentic Time Series Analysis in Evolving Environments

TimeSage-EV: A Live Benchmark for Agentic Time Series Analysis in Evolving Environments

URL: http://arxiv.org/abs/2608.14270v1
Date: Fri, 14 Aug 2026 12:53:21 GMT
Title: TimeSage-EV: A Live Benchmark for Agentic Time Series Analysis in Evolving Environments
Abstract summary: TimeSage-EV is a live benchmark for agentic time series analysis in evolving environments.<n>It tracks 60 real institutional scenarios across 6 domains, comprising 1,485 scenario-period QA pairs from Feb 2023 to May 2026.<n>TimeSage-EV evaluates state identification, data summarization, and outlook reasoning.
Score: 46.167956432673776
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: Time series analysis in high-stakes domains relies on recurring data releases, where new observations can alter the evidence base and the validity of later conclusions. Existing time series QA benchmarks mostly rely on fixed snapshots, leaving temporal validity and cutoff-aware evidence use unevaluated. We introduce TimeSage-EV, a live benchmark for agentic time series analysis in evolving environments. It tracks 60 real institutional scenarios across 6 domains, comprising 1,485 scenario-period QA pairs from Feb 2023 to May 2026 and spanning monthly, weekly, daily, and irregular release cadences. At each period, large language model (LLM) agents receive time series data and source reports, while the withheld target release provides ground truth. TimeSage-EV evaluates state identification, data summarization, and outlook reasoning. Experiments with frontier LLM agents and TimeSage-1.0, a novel self-evolving agent with a reusable analytical skill library, reveal significant performance gaps across model tiers and recurring failures in temporal validity, exogenous context use, and adaptation. We release TimeSage-EV as a research resource with monthly updates, code, a leaderboard, and failure-mode analyses.

Related papers

LiveHouse-TS: An Open-world Living Benchmark for Time Series Foundation Models [41.28223590438589]
We introduce LiveHouse-TS, the first open-world living benchmark infrastructure for Time Series Foundation Models.<n>LiveHouse-TS shifts time series benchmarking from snapshot accuracy to continuous temporal validity.<n>Our infrastructure serves as a continuous time series infrastructure designed to explore vital, long-term scientific questions.
arXiv Detail & Related papers (2026-08-18T02:50:48Z)
TimeSage-MT: A Multi-Turn Benchmark for Evaluating Agentic Time Series Reasoning [44.68126840122709]
Time series data inform critical decisions across many real-world domains.<n>It remains unclear whether large language model (LLM) agents can conduct reliable time series analysis across multi-turn conversations.<n>TimeSage-MT is a benchmark for agentic time series reasoning with 240 tasks and 2,680 dialogue turns across 8 real-world domains.
arXiv Detail & Related papers (2026-05-31T23:34:35Z)
Impermanent: A Live Benchmark for Temporal Generalization in Time Series Forecasting [8.050703449557028]
Impermanent is a live benchmark that evaluates forecasting models under open-world temporal change.<n>We focus on the top 400 repositories by star count and construct time series from issues opened, pull requests opened, push events, and new stargazers.<n>By shifting evaluation from static accuracy to sustained performance, Impermanent takes a concrete step toward assessing when and whether foundation-level generalization in time-series forecasting can be meaningfully claimed.
arXiv Detail & Related papers (2026-03-09T17:59:00Z)
TSAQA: Time Series Analysis Question And Answering Benchmark [85.35545785252309]
Time series data are integral to critical applications across domains such as finance, healthcare, transportation, and environmental science.<n>We introduce TSAQA, a novel unified benchmark designed to broaden task coverage and evaluate diverse temporal analysis capabilities.
arXiv Detail & Related papers (2026-01-30T17:28:56Z)
A Unified Frequency Domain Decomposition Framework for Interpretable and Robust Time Series Forecasting [81.73338008264115]
Current approaches for time series forecasting, whether in the time or frequency domain, predominantly use deep learning models based on linear layers or transformers.<n>We propose FIRE, a unified frequency domain decomposition framework that provides a mathematical abstraction for diverse types of time series.<n>Fire consistently outperforms state-of-the-art models on long-term forecasting benchmarks.
arXiv Detail & Related papers (2025-10-11T09:59:25Z)
Time-RA: Towards Time Series Reasoning for Anomaly with LLM Feedback [55.284574165467525]
Time-series Reasoning for Anomaly (Time-RA) transforms classical time series anomaly detection into a generative, reasoning-intensive task.<n>Also, we introduce the first real-world multimodal benchmark dataset, RATs40K, explicitly annotated for anomaly reasoning.
arXiv Detail & Related papers (2025-07-20T18:02:50Z)
GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation [90.53485251837235]
Time series foundation models excel in zero-shot forecasting, handling diverse tasks without explicit training. GIFT-Eval is a pioneering benchmark aimed at promoting evaluation across diverse datasets. GIFT-Eval encompasses 23 datasets over 144,000 time series and 177 million data points.
arXiv Detail & Related papers (2024-10-14T11:29:38Z)
A Closer Look at Debiased Temporal Sentence Grounding in Videos: Dataset, Metric, and Approach [53.727460222955266]
Temporal Sentence Grounding in Videos (TSGV) aims to ground a natural language sentence in an untrimmed video. Recent studies have found that current benchmark datasets may have obvious moment annotation biases. We introduce a new evaluation metric "dR@n,IoU@m" that discounts the basic recall scores to alleviate the inflating evaluation caused by biased datasets.
arXiv Detail & Related papers (2022-03-10T08:58:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.