TimeSage-MT: A Multi-Turn Benchmark for Evaluating Agentic Time Series Reasoning

• URL: http://arxiv.org/abs/2606.01498v1
• Date: Sun, 31 May 2026 23:34:35 GMT
• Title: TimeSage-MT: A Multi-Turn Benchmark for Evaluating Agentic Time Series Reasoning
• Abstract summary: Time series data inform critical decisions across many real-world domains.<n>It remains unclear whether large language model (LLM) agents can conduct reliable time series analysis across multi-turn conversations.<n>TimeSage-MT is a benchmark for agentic time series reasoning with 240 tasks and 2,680 dialogue turns across 8 real-world domains.
• Score: 44.68126840122709
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Time series data inform critical decisions across many real-world domains. While large language model (LLM) agents can analyze data through natural language and tools, it remains unclear whether they can conduct reliable time series analysis across multi-turn conversations. Existing benchmarks focus on single-step tasks such as forecasting and anomaly detection, overlooking practical workflows where user goals evolve, agents must build on prior analyses, and conclusions emerge from accumulated evidence. In this work, we introduce TimeSage-MT, a multi-turn benchmark for agentic time series reasoning with 240 tasks and 2,680 dialogue turns across 8 real-world domains, spanning basic exploration to decision-oriented analysis. TimeSage-MT is built through a reproducible pipeline that converts real-world time series data into multi-turn conversations with verifiable answers. It provides a unified evaluation protocol and public leaderboard for comparing time series agentic systems. To demonstrate the benchmark's utility, we evaluate frontier LLMs alongside TimeSage, a novel structured agent equipped with a comprehensive time series skill library. The results show sharp performance drops on decision-oriented tasks, driven by failures in memory, uncertainty handling, and domain-based decision making. TimeSage-MT exposes critical gaps in current agentic reasoning and provides a rigorous foundation for future development.

Related papers

• Harnessing Generalist Agents for Contextualized Time Series [67.28481985993467]
  Time series are often embedded in rich contexts that are essential for holistic modeling.<n>In this work, we introduce TimeClaw, an agentic harness framework for time series.<n>TimeClaw integrates executable temporal tools for grounded and auditable analysis, experience-driven capability evolution for creating reusable analytical routines, and episodic multimodal memory for retrieving relevant reasoning traces.
  arXiv  Detail & Related papers  (2026-06-03T20:20:34Z)
• TimelineReasoner: Advancing Timeline Summarization with Large Reasoning Models [54.737142557469916]
  We propose TimelineReasoner, a framework that shifts Timeline Summarization from static generation to an active, reasoning-driven process.<n>We show that TimelineReasoner significantly outperforms existing LLM-based TLS methods in terms of timeline accuracy, coverage, and coherence.<n>This work not only pushes the boundaries of TLS but also highlights the broader potential of LRM-based reasoning frameworks for timeline summarization.
  arXiv  Detail & Related papers  (2026-04-03T12:30:15Z)
• TSAQA: Time Series Analysis Question And Answering Benchmark [85.35545785252309]
  Time series data are integral to critical applications across domains such as finance, healthcare, transportation, and environmental science.<n>We introduce TSAQA, a novel unified benchmark designed to broaden task coverage and evaluate diverse temporal analysis capabilities.
  arXiv  Detail & Related papers  (2026-01-30T17:28:56Z)
• TS-Agent: A Time Series Reasoning Agent with Iterative Statistical Insight Gathering [16.95452463476229]
  We propose TS-Agent, a time series reasoning agent for large language models (LLMs)<n>Instead of mapping time series into text tokens, images, or embeddings, our agent interacts with raw numeric sequences through atomic operators.<n>Our experiments show that TS-Agent achieves performance comparable to state-of-the-art LLMs on understanding benchmarks.
  arXiv  Detail & Related papers  (2025-10-08T18:31:53Z)
• When LLM Meets Time Series: Can LLMs Perform Multi-Step Time Series Reasoning and Inference [12.867006554196358]
  We introduce the TSAIA Benchmark, a first attempt to evaluate Large Language Models as time-series AI assistants.<n>The benchmark encompasses a broad spectrum of challenges, ranging from constraint-aware forecasting to anomaly detection with threshold calibration.<n>We apply this benchmark to assess eight state-of-the-art LLMs under a unified evaluation protocol.
  arXiv  Detail & Related papers  (2025-09-01T22:58:57Z)
• Timing Analysis Agent: Autonomous Multi-Corner Multi-Mode (MCMM) Timing Debugging with Timing Debug Relation Graph [1.6392250108065922]
  Small metal pitches and increasing number of devices have led to longer turn-around-time for experienced human designers to debug timing issues.<n>Large Language Models (LLMs) have shown great promise across various tasks in language understanding and interactive decision-making.<n>We build a Timing Relation Graph (TDRG) that connects the reports with the relationships of debug traces from experienced timing engineers.
  arXiv  Detail & Related papers  (2025-04-15T04:14:36Z)
• Multi-modal Time Series Analysis: A Tutorial and Survey [36.93906365779472]
  Multi-modal time series analysis has emerged as a prominent research area in data mining.<n>However, effective analysis of multi-modal time series is hindered by data heterogeneity, modality gap, misalignment, and inherent noise.<n>Recent advancements in multi-modal time series methods have exploited the multi-modal context via cross-modal interactions.
  arXiv  Detail & Related papers  (2025-03-17T20:30:02Z)
• Time-MQA: Time Series Multi-Task Question Answering with Context Enhancement [55.2439260314328]
  Time Series Multi-Task Question Answering (Time-MQA) is a unified framework that enables natural language queries across multiple time series tasks.<n>Central to Time-MQA is the TSQA dataset, a large-scale dataset containing $sim $200k question-answer pairs.
  arXiv  Detail & Related papers  (2025-02-26T13:47:13Z)
• Position: What Can Large Language Models Tell Us about Time Series Analysis [69.70906014827547]
  We argue that current large language models (LLMs) have the potential to revolutionize time series analysis. Such advancement could unlock a wide range of possibilities, including time series modality switching and question answering.
  arXiv  Detail & Related papers  (2024-02-05T04:17:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.