BALMS: Benchmarking Agentic LLMs for Longitudinal Mental Health Sensing

• URL: http://arxiv.org/abs/2608.27219v1
• Date: Thu, 27 Aug 2026 15:00:17 GMT
• Title: BALMS: Benchmarking Agentic LLMs for Longitudinal Mental Health Sensing
• Abstract summary: We introduce BALMS, the first systematic benchmark of LLM-based agentic systems for longitudinal mental health sensing.<n>We find that zero-shot agents rarely outperform a simple mean baseline, except with stronger backbones or compact, semantically meaningful features. Chain-of-thought prompting improves reasoning-oriented backbones, but does not guarantee temporal grounding or numerical correctness.
• Score: 7.396595070882962
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Mental health assessment relies on episodic self-report scales, which convert subjective states such as stress into numerical scores but provide only sparse snapshots of wellbeing. Wearable devices offer longitudinal behavioral and physiological signals for continuous, low-burden monitoring. Recent LLM-driven personal-health agents enable natural language queries over wearable signals, but mainly handle short-term, retrieval-based lookups (e.g., highest step count over a week). They do not evaluate whether agents can reason over long-term signals to predict wellbeing scores paired with evidence-grounded rationales. To address this gap, we introduce BALMS, the first systematic benchmark of LLM-based agentic systems for longitudinal mental health sensing. BALMS spans 3 real-world longitudinal datasets, 2 task families (closed-form wellbeing-score prediction and rationale generation auto-graded by an LLM-as-Judge), 3 agentic paradigms evaluated across 5 open- and closed-source LLM backbones. We find that zero-shot agents rarely outperform a simple mean baseline, except with stronger backbones or compact, semantically meaningful features. Chain-of-thought prompting improves reasoning-oriented backbones, but does not guarantee temporal grounding or numerical correctness. Together with more analysis on efficiency and temporal scaling, BALMS highlights the need for longitudinal mental health agents that selectively retrieve history, ground temporal evidence, and reason over interpretable behavioral features.

Related papers

• MHGraphBench: Knowledge Graph-Grounded Benchmarking of Mental Health Knowledge in Large Language Models [13.221931392711406]
  Large language models (LLMs) are increasingly used in the mental health domain.<n>It remains unclear how well they capture related biomedical knowledge and how reliably they apply it to clinically salient structured judgments.<n>Here, we present a knowledge-graph-grounded benchmark for assessing LLMs on mental-health entity recognition, relation judgment, and two-hop reasoning.
  arXiv  Detail & Related papers  (2026-05-15T03:55:27Z)
• COTCAgent: Preventive Consultation via Probabilistic Chain-of-Thought Completion [1.3891530345631953]
  The Probabilistic Chain-of-Thought Completion Agent (COTCAgent) is a hierarchical reasoning framework for longitudinal electronic health records.<n>COTCAgent achieves 90.47% Top-1 accuracy on the self-built dataset and 70.41% on HealthBench, outperforming existing medical agents and mainstream large language models.
  arXiv  Detail & Related papers  (2026-05-14T16:17:35Z)
• EviCare: Enhancing Diagnosis Prediction with Deep Model-Guided Evidence for In-Context Reasoning [62.61394722212386]
  EviCare is an in-context reasoning framework that integrates deep model guidance into large language models.<n>It consistently outperforms both LLM-only and deep model-only baselines in experiments on two real-world EHR benchmarks.
  arXiv  Detail & Related papers  (2026-04-12T04:35:14Z)
• PREBA: Surgical Duration Prediction via PCA-Weighted Retrieval-Augmented LLMs and Bayesian Averaging Aggregation [51.96735866702332]
  PREBA is a retrieval-augmented framework that integrates PCA-weighted retrieval and Bayesian averaging aggregation.<n> PREBA significantly improves performance-for instance, reducing MAE by up to 40% and raising R2 from -0.13 to 0.62 over zero-shot inference.
  arXiv  Detail & Related papers  (2026-02-27T07:19:23Z)
• Benchmarking Early Deterioration Prediction Across Hospital-Rich and MCI-Like Emergency Triage Under Constrained Sensing [0.0]
  We present a leakage-aware benchmarking framework for early deterioration prediction.<n>We compare hospital-rich triage with a vitals-only, MCI-like setting, restricting inputs to information available within the first hour of presentation.
  arXiv  Detail & Related papers  (2026-02-09T09:32:49Z)
• From Conflict to Consensus: Boosting Medical Reasoning via Multi-Round Agentic RAG [65.0203623486525]
  Large Language Models (LLMs) exhibit high reasoning capacity in medical question-answering.<n>Their tendency to produce hallucinations and outdated knowledge poses critical risks in healthcare fields.<n>Existing methods rely on noisy token-level signals and lack the multi-round refinement required for complex reasoning.
  arXiv  Detail & Related papers  (2026-02-06T08:25:30Z)
• MIMIC-SR-ICD11: A Dataset for Narrative-Based Diagnosis [14.505360834752866]
  We introduce MIMIC-SR-ICD11, a large English diagnostic dataset built from EHR discharge notes and aligned to WHO ICD-11 terminology.<n>We present LL-Rank, a likelihood-based re-ranking framework that computes a length-normalized joint likelihood of each label given the clinical report context.
  arXiv  Detail & Related papers  (2025-11-07T18:55:22Z)
• Quantifying the Reasoning Abilities of LLMs on Real-world Clinical Cases [48.87360916431396]
  We introduce MedR-Bench, a benchmarking dataset of 1,453 structured patient cases, annotated with reasoning references.<n>We propose a framework encompassing three critical examination recommendation, diagnostic decision-making, and treatment planning, simulating the entire patient care journey.<n>Using this benchmark, we evaluate five state-of-the-art reasoning LLMs, including DeepSeek-R1, OpenAI-o3-mini, and Gemini-2.0-Flash Thinking, etc.
  arXiv  Detail & Related papers  (2025-03-06T18:35:39Z)
• An LLM-Powered Agent for Physiological Data Analysis: A Case Study on PPG-based Heart Rate Estimation [2.0195680688695594]
  Large language models (LLMs) are revolutionizing healthcare by improving diagnosis, patient care, and decision support through interactive communication.<n>We develop an LLM-powered agent for physiological time-series analysis aimed to bridge the gap in integrating LLMs with well-established analytical tools.<n>Built on the OpenCHA, our agent powered by OpenAI's GPT-3.5-turbo model features an orchestrator that embeds user interaction, data sources, and analytical tools to generate accurate health insights.
  arXiv  Detail & Related papers  (2025-02-18T13:09:59Z)
• LlaMADRS: Prompting Large Language Models for Interview-Based Depression Assessment [75.44934940580112]
  This study introduces LlaMADRS, a novel framework leveraging open-source Large Language Models (LLMs) to automate depression severity assessment.<n>We employ a zero-shot prompting strategy with carefully designed cues to guide the model in interpreting and scoring transcribed clinical interviews.<n>Our approach, tested on 236 real-world interviews, demonstrates strong correlations with clinician assessments.
  arXiv  Detail & Related papers  (2025-01-07T08:49:04Z)
• WellDunn: On the Robustness and Explainability of Language Models and Large Language Models in Identifying Wellness Dimensions [46.60244609728416]
  Language Models (LMs) are being proposed for mental health applications where the heightened risk of adverse outcomes means predictive performance may not be a litmus test of a model's utility in clinical practice. We introduce an evaluation design that focuses on the robustness and explainability of LMs in identifying Wellness Dimensions (WDs) We reveal four surprising results about LMs/LLMs.
  arXiv  Detail & Related papers  (2024-06-17T19:50:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.