HEARTS: Benchmarking LLM Reasoning on Health Time Series

• URL: http://arxiv.org/abs/2603.06638v1
• Date: Wed, 25 Feb 2026 05:23:23 GMT
• Title: HEARTS: Benchmarking LLM Reasoning on Health Time Series
• Abstract summary: We introduce HEARTS (Health Reasoning over Time Series), a unified benchmark for evaluating hierarchical reasoning capabilities of large language models (LLMs) over general health time series.<n> HEARTS integrates 16 real-world datasets across 12 health domains and 20 signal modalities, and defines a comprehensive taxonomy of 110 tasks grouped into four core capabilities.<n> evaluating 14 state-of-the-art LLMs on more than 20K test samples reveals intriguing findings.
• Score: 19.03077523786892
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The rise of large language models (LLMs) has shifted time series analysis from narrow analytics to general-purpose reasoning. Yet, existing benchmarks cover only a small set of health time series modalities and tasks, failing to reflect the diverse domains and extensive temporal dependencies inherent in real-world physiological modeling. To bridge these gaps, we introduce HEARTS (Health Reasoning over Time Series), a unified benchmark for evaluating hierarchical reasoning capabilities of LLMs over general health time series. HEARTS integrates 16 real-world datasets across 12 health domains and 20 signal modalities, and defines a comprehensive taxonomy of 110 tasks grouped into four core capabilities: Perception, Inference, Generation, and Deduction. Evaluating 14 state-of-the-art LLMs on more than 20K test samples reveals intriguing findings. First, LLMs substantially underperform specialized models, and their performance is only weakly related to general reasoning scores. Moreover, LLMs often rely on simple heuristics and struggle with multi-step temporal reasoning. Finally, performance declines with increasing temporal complexity, with similar failure modes within model families, indicating that scaling alone is insufficient. By making these gaps measurable, HEARTS provides a standardized testbed and living benchmark for developing next-generation LLM agents capable of reasoning over diverse health signals.

Related papers

• TimeSRL: Generalizable Time-Series Behavioral Modeling via Semantic RL-Tuned LLMs -- A Case Study in Mental Health [10.41600839007471]
  TimeSRL is a framework that routes predictions through an explicit semantic bottleneck.<n>It achieves state-of-the-art performance on a benchmark designed to stress-test cross-cohort generalization.<n>TimeSRL significantly outperforms prior methods in cross-benchmark transfer across different sensing pipelines.
  arXiv  Detail & Related papers  (2026-05-20T15:25:46Z)
• TSAQA: Time Series Analysis Question And Answering Benchmark [85.35545785252309]
  Time series data are integral to critical applications across domains such as finance, healthcare, transportation, and environmental science.<n>We introduce TSAQA, a novel unified benchmark designed to broaden task coverage and evaluate diverse temporal analysis capabilities.
  arXiv  Detail & Related papers  (2026-01-30T17:28:56Z)
• An Expert-grounded benchmark of General Purpose LLMs in LCA [1.9645069537947935]
  Large language models (LLMs) are increasingly being explored as tools to support life cycle assessment (LCA)<n>This study provides the first expert-grounded benchmark of LLMs in LCA.
  arXiv  Detail & Related papers  (2025-10-22T15:56:54Z)
• SciTS: Scientific Time Series Understanding and Generation with LLMs [53.35994674187729]
  We introduce SciTS, a benchmark spanning 12 scientific domains and 43 tasks.<n>We benchmark 17 models, including text-only LLMs, multimodal LLMs, and unified time series models.<n>We then introduce Time Omni, a framework that equips LLMs with the ability to understand and generate time series.
  arXiv  Detail & Related papers  (2025-09-26T09:25:16Z)
• Enhancing LLM Reasoning for Time Series Classification by Tailored Thinking and Fused Decision [8.256998757769322]
  ReasonTSC is a framework designed to leverage LLM reasoning for time series classification.<n>It steers the model to think over the essential characteristics of time series data.<n>It integrates predictions and confidence scores from plug-in classifiers, e.g., domain-specific time series models, as in-context examples.
  arXiv  Detail & Related papers  (2025-06-01T03:15:54Z)
• General-Reasoner: Advancing LLM Reasoning Across All Domains [64.70599911897595]
  Reinforcement learning (RL) has recently demonstrated strong potential in enhancing the reasoning capabilities of large language models (LLMs)<n>We propose General-Reasoner, a novel training paradigm designed to enhance LLM reasoning capabilities across diverse domains.<n>We train a series of models and evaluate them on a wide range of datasets covering wide domains like physics, chemistry, finance, electronics etc.
  arXiv  Detail & Related papers  (2025-05-20T17:41:33Z)
• Benchmarking Spatiotemporal Reasoning in LLMs and Reasoning Models: Capabilities and Challenges [4.668749313973097]
  This paper systematically evaluate Large Language Models (LLMs) and Large Reasoning Models (LRMs) across three levels of reasoning complexity.<n>We curate 26 challenges where models answer directly or by Python Code Interpreter.<n>LRMs show robust performance across tasks with various levels of difficulty, often competing or surpassing traditional first-principle-based methods.
  arXiv  Detail & Related papers  (2025-05-16T18:32:35Z)
• Reasoning Capabilities and Invariability of Large Language Models [49.23570751696334]
  We aim to provide a comprehensive analysis of Large Language Models' reasoning competence.<n>We introduce a new benchmark dataset with a series of simple reasoning questions demanding shallow logical reasoning.<n>An empirical analysis involving zero-shot and few-shot prompting across 24 LLMs of different sizes reveals that, while LLMs with over 70 billion parameters perform better in the zero-shot setting, there is still a large room for improvement.
  arXiv  Detail & Related papers  (2025-05-01T18:12:30Z)
• BRIDGE: Benchmarking Large Language Models for Understanding Real-world Clinical Practice Text [14.409097921305134]
  BRIDGE is a comprehensive benchmark comprising 87 tasks sourced from real-world clinical data sources across nine languages.<n>It covers eight major task types spanning the entire continuum of patient care across six clinical stages and 20 representative applications.<n>Our results reveal substantial performance variation across model sizes, languages, natural language processing tasks, and clinical specialties.
  arXiv  Detail & Related papers  (2025-04-28T04:13:18Z)
• TransientTables: Evaluating LLMs' Reasoning on Temporally Evolving Semi-structured Tables [47.85408648193376]
  Large language models (LLMs) are typically trained on static datasets, limiting their ability to perform effective temporal reasoning.<n>We present the TRANSIENTTABLES dataset, which comprises 3,971 questions derived from over 14,000 tables, spanning 1,238 entities across multiple time periods.
  arXiv  Detail & Related papers  (2025-04-02T16:34:43Z)
• LLM-PS: Empowering Large Language Models for Time Series Forecasting with Temporal Patterns and Semantics [56.99021951927683]
  Time Series Forecasting (TSF) is critical in many real-world domains like financial planning and health monitoring.<n>Existing Large Language Models (LLMs) usually perform suboptimally because they neglect the inherent characteristics of time series data.<n>We propose LLM-PS to empower the LLM for TSF by learning the fundamental textitPatterns and meaningful textitSemantics from time series data.
  arXiv  Detail & Related papers  (2025-03-12T11:45:11Z)
• Empowering Time Series Analysis with Large Language Models: A Survey [24.202539098675953]
  We provide a systematic overview of methods that leverage large language models for time series analysis. Specifically, we first state the challenges and motivations of applying language models in the context of time series. Next, we categorize existing methods into different groups (i.e., direct query, tokenization, prompt design, fine-tune, and model integration) and highlight the key ideas within each group.
  arXiv  Detail & Related papers  (2024-02-05T16:46:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.