WearableQA: A Benchmark for Health Reasoning over Real-World Wearable Data

• URL: http://arxiv.org/abs/2609.05405v1
• Date: Fri, 04 Sep 2026 17:52:37 GMT
• Title: WearableQA: A Benchmark for Health Reasoning over Real-World Wearable Data
• Abstract summary: WearableQA is a benchmark for evaluating AI systems' reasoning over a real user's longitudinal wearable record.<n>We introduce 16 question types organized along two complementary axes: data versus health reasoning, and single- versus cross-signal reasoning.<n> Evaluation of 14 proprietary and open-source LLMs demonstrates that WearableQA effectively differentiates model capabilities, with performance ranging from 19.6% to 72.9% against a 10% chance baseline.
• Score: 37.47499553332249
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in wearable sensing enable continuous monitoring of physiological and behavioral signals, yet existing benchmarks rarely evaluate whether AI systems can reason over a real user's longitudinal wearable record. We introduce WearableQA, a benchmark comprising 4,084 10-option multiple-choice questions constructed from the wearable time series, blood biomarkers, and demographics of 200 real users, each with up to 500 days of daily measurements. WearableQA preserves authentic wearable distributions that include device noise and inter-individual variability. To evaluate distinct reasoning capabilities, we introduce 16 question types organized along two complementary axes: data versus health reasoning, which distinguishes computation over longitudinal measurements from physiological interpretation; and single- versus cross-signal reasoning, which separates reasoning about individual signals from the integration of multiple signals. To construct reliable questions at scale, we adopt a dual-grounding framework that combines literature-grounded physiological findings with statistically validated population-grounded physiological patterns. This enables the capture of meaningful relationships observed in real-world wearable data. Evaluation of 14 proprietary and open-source LLMs demonstrates that WearableQA effectively differentiates model capabilities, with performance ranging from 19.6% to 72.9% against a 10% chance baseline. Moreover, WearableQA remains far from solved: most models achieve accuracies below 60%. Overall, WearableQA provides a realistic and diagnostic benchmark for evaluating LLM reasoning over real-world wearable data.

Related papers

• NovGauge: A Fine-Grained Benchmark for Diagnosing LLMs' Capability in Paper Novelty Assessment [54.4265627247104]
  Large language models (LLMs) are increasingly used in peer review at major AI conferences.<n>Existing benchmarks assess novelty as a single holistic score.<n>We present NovGauge, a human-anchored benchmark for fine-grained novelty assessment diagnosis.
  arXiv  Detail & Related papers  (2026-09-10T08:34:56Z)
• HealthLoopQA: A Context-Aware Question Answering Benchmark for Interpreting Wearable Monitoring Data in Diabetes Care [25.340084694880048]
  We introduce HealthLoopQA, a diagnostic benchmark for evaluating large language models (LLMs) over continuous diabetes monitoring data.<n>HealthLoopQA comprises 127 tasks and over 1,500 QA instances spanning process mining, anomaly detection, and prediction over 30-day horizons.<n>We complement real-world datasets with a fault-injected simulation testbed modeling diverse device malfunctions and cyber-physical attacks to generate physiologically plausible hazard scenarios.
  arXiv  Detail & Related papers  (2026-09-07T03:11:34Z)
• Rethinking the Evaluation and Optimization of LLM-Based Social Simulation [56.18577997572273]
  We use the subjectivity coefficient to analyze how accuracy-based evaluation and hard-label training fail as subjectivity grows.<n>We construct SUBJSIM, a benchmark of 19,300 contexts covering 193 annotators and 100 subjective questions.<n>Results on SUBJSIM demonstrate the advantages of our method.
  arXiv  Detail & Related papers  (2026-08-20T06:26:03Z)
• Full-Self Diagnostics (FSD): Physics-Grounded Visual Biomarker Inference from Smartphone Video via Inverse Problems and Operator Learning [0.0]
  Full-Self Diagnostics (FSD) is a unified mathematical framework for recovering latent physiological states from unconstrained 9-second facial videos.<n> Empirical validation on 38812 real-world paired scans across 59 subjects demonstrates practical performance.<n>Results confirm that consumer-grade facial video encodes sufficient structured information for clinically relevant, non-invasive inference under fully unconstrained conditions.
  arXiv  Detail & Related papers  (2026-06-11T20:26:47Z)
• Towards a General Intelligence and Interface for Wearable Health Data [45.854819784803055]
  We propose a foundation model for wearable health that is pretrained on more than one trillion minutes of unlabeled sensor signals.<n>We find that this population scale representation unlocks label-efficient few-shot learning and generative capabilities for robust daily metric estimation.
  arXiv  Detail & Related papers  (2026-05-21T17:24:06Z)
• ESL-Bench: An Event-Driven Synthetic Longitudinal Benchmark for Health Agents [10.567096101127886]
  ESLBench is an event-driven synthesis framework providing 100 synthetic users.<n>We evaluate 13 methods spanning LLMs with tools, DB-native agents, and memory-augmented RAG.
  arXiv  Detail & Related papers  (2026-04-03T07:55:56Z)
• A Neuro-Symbolic System for Interpretable Multimodal Physiological Signals Integration in Human Fatigue Detection [3.430778683859764]
  We propose a neuro-symbolic architecture that learns four interpretable physiological concepts, oculomotor dynamics, gaze stability, prefrontal hemodynamics, and multimodal, from eye-tracking and neural hemodynamics, functional near-infrared spectroscopy, (fNIRS) windows using attention-based encoders, and combines them with differentiable approximate reasoning rules using learned weights and soft thresholds.<n>We apply this system to fatigue classification from multimodal physiological signals, a domain that requires models that are accurate and interpretable, with internal reasoning that can be inspected for safety-critical use.
  arXiv  Detail & Related papers  (2026-03-25T14:41:02Z)
• Medical Imaging AI Competitions Lack Fairness [50.895929923643905]
  We assess fairness along two complementary dimensions: whether challenge datasets are representative of real-world clinical diversity, and whether they are accessible and legally reusable in line with the FAIR principles.<n>Our findings show substantial biases in dataset composition, including geographic location, modality, and problem type-related biases, indicating that current benchmarks do not adequately reflect real-world clinical diversity.<n>These shortcomings expose foundational limitations in our benchmarking ecosystem and highlight a disconnect between leaderboard success and clinical relevance.
  arXiv  Detail & Related papers  (2025-12-19T13:48:10Z)
• Toward Foundation Model for Multivariate Wearable Sensing of Physiological Signals [2.370585289844609]
  We propose the first multi-modal and ubiquitous foundation model designed to extract informative representations from wearable sensing data.<n>Specifically, we design a channel-aware attention mechanism with a shared special liaison token to detect signal patterns in both intra-sensor and inter-sensors.<n>Our model shows exceptional generalizability across 11 public wearable sensing datasets, spanning 18 applications in mental health, body state inference, vital sign estimation, and disease risk evaluation.
  arXiv  Detail & Related papers  (2024-12-12T23:35:18Z)
• Video-based Remote Physiological Measurement via Cross-verified Feature Disentangling [121.50704279659253]
  We propose a cross-verified feature disentangling strategy to disentangle the physiological features with non-physiological representations. We then use the distilled physiological features for robust multi-task physiological measurements. The disentangled features are finally used for the joint prediction of multiple physiological signals like average HR values and r signals.
  arXiv  Detail & Related papers  (2020-07-16T09:39:17Z)
• Hemogram Data as a Tool for Decision-making in COVID-19 Management: Applications to Resource Scarcity Scenarios [62.997667081978825]
  COVID-19 pandemics has challenged emergency response systems worldwide, with widespread reports of essential services breakdown and collapse of health care structure. This work describes a machine learning model derived from hemogram exam data performed in symptomatic patients. Proposed models can predict COVID-19 qRT-PCR results in symptomatic individuals with high accuracy, sensitivity and specificity.
  arXiv  Detail & Related papers  (2020-05-10T01:45:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.